Major server performance problems after "upgrade"
Hi, I posted this in my host's forum but haven't had any response so I'm reposting here. This problem is in regards to a dedicated server that I'm leasing.-------
So I've been running a server with RH9 + Plesk 6 for a couple years now. I know, it was getting a bit old, but I kept it up to date with yum+ART and it was working just fine really.
Well the other night I pulled the most boneheaded move I've pulled in years: an rm -rf * in the filesystem root. Thought I was in a different directory. I about crapped myself when I got a warning about not being able to remove /proc. Lucky for me, I killed it before it got any important data...it just wiped out /bin, /sbin, and a few other choice pieces of the OS.
So I ordered a system restore and suddenly had a brand new system with RHEL and Plesk 7.5.4. Cool, at least maintenence should be easier from here on out, etc. Got all my data back on the server. All is good....almost.
The problem is that this thing is a dog by comparison. I used to have server loads in the 0.3 - 0.8 through most of the day. Occasionally it'd hop over 1 for a little bit. Now my server load is over 1 a *lot*. CPU usage is up a *lot* - both mysqld and httpd are using a lot more cpu. The thing is just slow, and I can't figure it out. For instance, there is one forum on this server and loading the index used to take between 0.2 and 0.5 seconds, no big deal. But on the new server it will frequently (like say once every 8-10 loads), take 3-8 seconds. Just an example.
I was running Apache 2, PHP 4.3.11, and Mysql 4.1 on the old server (via ART packages). It now appears as though I'm pretty much running the same set, so I'm confused.
Here's what I've done:
1) Enabled query caching in Mysql, and basically setup my config the way it was before, which was working well. Verified that qcache is working and has plenty of spare memory.
2) Tuned apache & php to settings similar to what they were. Disabled some modules that weren't in my old config and not necessary, like ldap auth. Installed mmcache and configured it the way it was before (disabled the script cache that was installed by default, which didn't seem to work well). Verified the cache is working and has plenty of spare memory.
3) Got rid of some new processes that I don't want, like dr.web antivirus...and i'm about to get rid of spamassassin since there are 8 spamd processes running and my plesk license doesn't allow me to use em.
One thing I noticed is that apache must be running a different MPM than my old config, or something. I used to see only one httpd process in top, now I see a half dozen of them, and I see them going defunct pretty regularly (although some people have said it's normal) which is something I've never seen before. These apache processes take up a lot more CPU than what I was used to before. I don't know a hell of a lot about apache configuration.
Nothing out of the ordinary in messages, apache error logs, or any other error logs.