John Hesch

Two weeks ago my server was used to attempt break-ins of at least three other servers. Before this happened I was totally unaware of what was needed to secure a web server. I'm still not close to being a security expert but I have learned quite a bit in the past few days. In this post I hope to provide the uninformed or newby server administrator with information and tools to secure¹ their server against hackers.

My experience began on a Friday evening when I checked my inbox and found several emails from other server administrators, suggesting that my server had been compromised and was being used to attempt break-ins of their systems.

The first thing I did was log into WHM² and checked the Show Current CPU Usage link. I noticed that the CPU was pegged at 100% by several Nobody³ users. I should of taken the time to record exactly what processes were being used but I immediately killed all Nobody processes. This action brought the server back to normal operation.

I honestly did not know how to proceed. I was under the assumption that since I had been operating this server for several years without incident, that I had a fairly secure server. I started with Googling Apache security topics and visited the discussion forums hosted by the company where I lease my dedicated server.

The two applications that I consistently read about were APF (Advanced Policy Firewall) and BFD (Brute Force Detection). I've always used a firewall on my personal PC but it never occurred to me that I would need a firewall for my server.

I've never installed programs on the server before so this was a new learning process. Luckily I found a great website that took me step by step through the process.

Once I got AFP and BFD installed I began searching for the culprit. I learned that most rootkits are stored and run from /tmp folders, so I ran the following commands from SSH as a root user.

When I got to the /var/tmp/ directory I found what I was looking for, a suspicious directory called /bebe/⁴. In this directory were several files beginning with an IP subset.pscan.22. I'm assuming the IP subsets were the IP addresses my system was scanning and the port was 22.

I zipped up the files, downloaded a copy and then deleted the directory.

From what I have read, once a rootkit is uploaded to your system the hard drive should be formatted followed by a fresh install of the OS. I took several additional steps and I feel fairly confident that I'm now running a clean system. I'll follow up with another post on what additional steps I've taken, and tell you what my daily security routine involves.

1. From what I have read there is no foolproof way to keep all hackers out of my server. There are ways to prevent the average to above average hacker from breaking in. [back]
2. WebHost Manager is the control center of the cPanel / WebHost Manager package. It is used to set up and manage accounts, monitor bandwidth and services, and much more - all aspects of web hosting are covered through WebHost Manager. [back]
3. I'm not really sure what the Nobody user is but some processes need to be run by Apache without passwords. Nobody is a username that doesn't require a password. [back]
4. It took me awhile to figure out how to get into the directory as /bebe/ was preceded with several spaces. [back]