Page 1 of 1

Preventing bots from hammering website

Posted: Sat Feb 09, 2019 1:44 am
by NotABulbasaur
Joomla 7.9.2, PHP 7.2.11. FTP/SSH access.

In recent weeks, I've noticed an increase of bot traffic. Adding a crawl-delay of 10 to robots.txt helped to some extent, but once or twice a day I get various misbehaving ones that hit the site fast enough to spike the memory usage, which sets the webhost's process-killer off for a few minutes. (Shared hosting, unfortunately. Site's not actually mine, so upgrading hosting isn't really an option. To be honest, aside from this issue we haven't had many problems with them.)

I'm looking into ways to prevent this from happening. I've blocked a few of the misbehaving bots with .htaccess, although that doesn't help when new ones show up. IP bans seem to have the same issue (plus IPs can be spoofed).

I've looked into Corz's anti-hammer, but it requires me to set permissions to 777, which is a trade-off I'm not entirely comfortable with.

There are paid extensions to help with this, but I'd rather not ask the website owner to shell out for one if there are other options.

Any other suggestions? Thanks in advance.

Re: Preventing bots from hammering website

Posted: Sat Feb 09, 2019 5:17 pm
by Webdongle