Spiders Causing Excessive Server Load

Today someone brought my server load to more than 22 while attempting to download various forums with a spider program. It took me a while to figure out what was going on and limit the access for his IP address's.

What can I do in the furture to either automatically ban IP addresses that cause excessive server load, or prevent people from spidering the forums (I already have a robots file set up).

 

 

 

 

Top