It’s been a crazy week. Apparently my last post struck a nerve. :)
After a momentus day which saw the Supreme Court legalize marriage equality, I was thinking about what could potentially follow in regards to state politics. Knowing many of Roy Moore’s statements and actions in the months leading up to the ruling, I mentally composed a letter to him while I was working out at the gym. I typed it up that evening and posted it. I shared it on Facebook and Twitter, made a few edits and went to bed.
I woke up the next morning to an email box full of messages and comments. I checked my analytics platform, and found that that post had absolutely rocketed off in a way I was not even remotely expecting to happen.
My blog usually gets about 7,000 pageviews a month. A few hundred a day, mostly to a few articles I’ve published over the years that have been linked to Stack Overflow and some other places. I don’t even update that frequently, although I’ve set a goal for myself to publish at least one post a month now.
At the peak of the traffic to the Roy Moore article on Monday, I was doing my monthly totals each hour. In total, the letter was viewed over 100,000(!) times. So this is some notes on how I scaled a tiny blog running on 10-year-old hardware to handle a “viral” load - and how I handled the load myself.
Hardware
This site is served off a Dell Poweredge SC1425, a server of roughly 2005 vintage. Yes, it’s quite old, but it works and usually serves my needs fine. That is, it serves this blog and a few other domains, handles my email and serves as a repository for my programming projects. That said, it is due for replacement later this year as funds allow.
Architecture
For years this blog ran on Wordpress, but a couple years ago I decided to dump Wordpress for Jekyll. Jekyll is a program that generates static HTML from Markdown. So, there’s no database. There’s no code that needs to be executed to build a page. All the pages are pre-built when the site is generated and served as static pages by Apache.
My primary reason for dropping Wordpress was I simply got tired of maintaining it. Having to constantly patch it up to avoid its numerous security holes in fear of hacks for what is just kind of a side hobby got tiresome. Jekyll, with its static pages, has a very low attack surface.
Because all the pages are pre-generated, it is literally no work for Apache to handle these requests. They’re super fast and Apache could handle thousands of these at any time, even on my ancient hardware. While load wasn’t the original reason I moved to Jekyll, it was invaluable when the time came.
CDN
But when it became clear that this was really taking off, I began to worry about bandwidth. My server is hosted by such an arrangement as that I don’t pay for bandwidth, but I do not want to become a burden on my provider. So Saturday when the traffic started to spike, I moved the site to Cloudflare.
Cloudflare is a “free” CDN. They’re actually a CDN by accident; their primary mission is threat protection. They can cache copies of your site around the world and serve them directly from their servers, monitoring and stopping threats. But because of that, you get a CDN by default. And they have a very very friendly free tier that was great for my needs.
And for my site, this is perfect. It’s all static content that barely if ever changes. Comments are delivered via Javascript from Disqus and rendered in the browser so, from a cache perspective, the page never changes. Very very easy to forward cache.
This was quite possibly the best thing I did, and it took almost no time to set up and get working. Maybe 15 minutes max, but a few days for the changes to delegate across the Internet to everywhere in the world. By the time the peak traffic hit on Monday, 90% of my site’s requests were being served by Cloudflare. Very few requests were actually hitting my server and what was was at the level of normal traffic. The site stayed up and was snappy fast.
Wetware
Here’s where I discuss my head. :)
I’ll be honest, all the attention that I didn’t expect has been stressful and kind of distracting. It’s been picked up and syndicated by newspapers across the state. I’ve given a few interviews on the letter and turned down or just flat out didn’t respond to a few others (mostly ones that I could tell were just going to attack me without any interest in a civilized discussion of the facts.)
I’m also not used to having to take such an active role in moderating my site. Cleaning up the comments and insuring the discussion stays on track has been a bit taxing. Also reading hundreds of email messages and comments takes time. And while the very, very vast majority of them were supportive or at least civil, there are always a few people who prove that John Gabriel’s Greater Internet F-wad Theory exists for a reason. I could have done without the hatemail, but thankfully, it was such a tiny amount that it was easy to ignore.
This is actually not my first brush with “Internet fame.” Back in 2001, shortly after the 9/11 attacks, I was so moved by the world’s response to it that I crafted a short little Flash video saying thanks, threw it up on my webspace at Auburn and send it to a couple of people. It spread all over the world and within a few days the traffic crashed Auburn’s web server. I later found out that they had to move my account to a new machine solely to keep up with all the traffic it generated.
Even then, I got hate mail over that. Really. I guess some people are just miserable and want others to be miserable too. Thankfully, I have the mental facilities now to deal with this type of thing that I didn’t have then, or even a few months ago. The cognitive behavioral therapy I’ve had over the last year was immensely helpful in helping me weather the storm.
I also have a longstanding policy of not reading the comments on major news outlets. And I’m not the only one. So you won’t find me losing sleep over anonymous comments on the Internet, or even reading them at all. I have a browser plugin that blocks them out entirely.
With all this going on, it’s been difficult to focus on the things I needed to get done this week. I think this may have been the toughest and most frustrating part of this whole experience to deal with.
I still haven’t heard from the one person I do want to hear from, though: Roy Moore.
Conclusions
Traffic is returning now to it’s normal levels. People have moved on to whatever the next big outrage or Internet cat video is. And I guess I can go back to writing about Apple and programming and other fun things to my audience of a handful of people. But if you do want to stick around and read my musings on tech and programming, you would be welcome.
A number of people have suggested I run for public office. I’m flattered, but politics is not really interesting to me in such a way as that I would actually like to take part in it. Maybe I will run for city council, though. I really want the potholes in front of my house fixed properly for once. :)
More than anything, I stand in awe of the modern world we live in. In the grand scheme of things, I’m a nobody. I’m just some guy with a little tech blog writing about life in dot-com. And yet, a hundred thousand people saw what I wrote. Major newspapers picked it up and ran it. And maybe, just maybe, it might make a damn difference.
The democratization of communications is one of the crowning achievements of the Internet. Something like this would not have even been remotely possible even twenty years ago. Now, thanks to the Internet, anyone can get their voice out there and heard.
And it wouldn’t have been possible at all without all my friends and all the awesome people around the state and around the world sharing it. Thank you for sharing my letter around. Thank you all for making this a memorable week for me, and I’m glad my little letter resonated with you enough to share it.
But why? Why did this particular expression of exasperation strike such a nerve? I think it’s because of the duality of being a Southerner is a common bond among all of us who live here. Living with such a checkered history but knowing that that history doesn’t accurately reflect how things are now. We all feel it, even if we don’t know how to put words to it. Perhaps the Bitter Southerner’s Manifesto explains it best:
You see, the South is a curiosity to people who aren’t from here. Always has been. Open up your copy of Faulkner’s 1936 masterpiece, “Absalom, Absalom!” Find the spot where Quentin Compson’s puzzled Canadian roommate at Harvard says to him, “Tell about the South. What it’s like there. What do they do there. Why do they live there. Why do they live at all.”
It always comes down to that last bit: With all our baggage, how do we live at all? A lot of people in the world believe that most folks in the South are just dumb. Or backward. Just not worth their attention.
…
But there is another South, the one that we know: a South that is full of people who do things that honor genuinely honorable traditions. Drinking. Cooking. Reading. Writing. Singing. Playing. Making things. It’s also full of people who face our region’s contradictions and are determined to throw our dishonorable traditions out the window. The Bitter Southerner is here for Southern people who do cool things, smart things, things that change the whole world, or just a few minds at a time.
