Joomla! Discussion Forums

I have 3 sites running on my webserver that seem to be showing signs of some attempted exploit. All 3 are Joomla 1.0.15 sites, 2 with VirtueMart running 1 not. I notice it when nagios starts reporting close to 300 processes running on my server and I take a look at another real-time reporting tool and it will report 125-130 visitors at a site at one time. When I open up the log files here is what I am finding:

Site #1:
74.208.147.140 - - [24/Oct/2009:04:04:40 -0600] "HEAD /contact-us/ruminate-magazine.html HTTP/1.1" 404 373 "5be9b90b912a4575cb0fcaf7de1d484f" "-"

Site #2:
74.208.198.241 - - [24/Oct/2009:13:21:26 -0600] "HEAD /images/comprofiler/tnnull HTTP/1.1" 404 410 "5be9b90b912a4575cb0fcaf7de1d484f" "-"

Site #3:
74.208.147.140 - - [24/Oct/2009:04:02:36 -0600] "HEAD /news/camo-patterns.html HTTP/1.1" 404 373 "5be9b90b912a4575cb0fcaf7de1d484f" "-"

Each one of these will be listed several thousand times over a 15 minute period. I am not expert when it comes to apache but the thing that I find interesting is that the second to the last field in all 3 is the same "5be9b90b912a4575cb0fcaf7de1d484f" and I can't find any info relating to that, is it a cookie file! Also each call is being made from it's own IP address, which almost suggests some kind of possible SQL Injection attack. I have both mod_security and mod_evasive installed but it isn't helping. There aren't enough HTTP calls to trigger mod_evasive, and the problem isn't the number of calls it is that they are spawning processes which are eating up resources. I have trimmed the 'KeepAliveTimeout' on apache to 2 seconds but that isn't helping either.

Anyone have any clue as to where I should start looking?

and have you blocked the IP range from your server - or asked your host to do this?

I can't block the IP range, it is the IP of the websites. Can't really firewall your own IP address ;-)

ok, then scrub that idea, at least it will stop your sql attacks!
have you run an optimisation on your database. or tracked down what can be running wild with the " top " command

Database tables are optimized every hour via cron. It's a bunch of apache processes. The log files are full of HTTP calls that are being made by the site's own IP address. They are not frequent enough to trigger mod_evasive though, I am getting about 4 per second. Of course each call is triggering a process ....

I am curios about the last field in the HTTP access.log though. Why all 3 sites would have the same value, and I am not sure what that value represents.

Here again is a sample from the access.log for one of the domains:

74.208.198.241 - - [24/Oct/2009:13:13:06 -0600] "HEAD /images/comprofiler/tnnull HTTP/1.1" 404 410 "5be9b90b912a4575cb0fcaf7de1d484f" "-"
74.208.198.241 - - [24/Oct/2009:13:13:06 -0600] "HEAD /images/comprofiler/tnnull HTTP/1.1" 404 410 "5be9b90b912a4575cb0fcaf7de1d484f" "-"
74.208.198.241 - - [24/Oct/2009:13:13:06 -0600] "HEAD /images/comprofiler/tnnull HTTP/1.1" 404 410 "5be9b90b912a4575cb0fcaf7de1d484f" "-"
74.208.198.241 - - [24/Oct/2009:13:13:06 -0600] "HEAD /images/comprofiler/tnnull HTTP/1.1" 404 410 "5be9b90b912a4575cb0fcaf7de1d484f" "-"
74.208.198.241 - - [24/Oct/2009:13:13:07 -0600] "HEAD /images/comprofiler/tnnull HTTP/1.1" 404 410 "5be9b90b912a4575cb0fcaf7de1d484f" "-"
74.208.198.241 - - [24/Oct/2009:13:13:07 -0600] "HEAD /images/comprofiler/tnnull HTTP/1.1" 404 410 "5be9b90b912a4575cb0fcaf7de1d484f" "-"
74.208.198.241 - - [24/Oct/2009:13:13:07 -0600] "HEAD /images/comprofiler/tnnull HTTP/1.1" 404 410 "5be9b90b912a4575cb0fcaf7de1d484f" "-"
74.208.198.241 - - [24/Oct/2009:13:13:07 -0600] "HEAD /images/comprofiler/tnnull HTTP/1.1" 404 410 "5be9b90b912a4575cb0fcaf7de1d484f" "-"
74.208.198.241 - - [24/Oct/2009:13:13:08 -0600] "HEAD /images/comprofiler/tnnull HTTP/1.1" 404 410 "5be9b90b912a4575cb0fcaf7de1d484f" "-"
74.208.198.241 - - [24/Oct/2009:13:13:08 -0600] "HEAD /images/comprofiler/tnnull HTTP/1.1" 404 410 "5be9b90b912a4575cb0fcaf7de1d484f" "-"
74.208.198.241 - - [24/Oct/2009:13:13:08 -0600] "HEAD /images/comprofiler/tnnull HTTP/1.1" 404 410 "5be9b90b912a4575cb0fcaf7de1d484f" "-"
74.208.198.241 - - [24/Oct/2009:13:13:08 -0600] "HEAD /images/comprofiler/tnnull HTTP/1.1" 404 410 "5be9b90b912a4575cb0fcaf7de1d484f" "-"
74.208.198.241 - - [24/Oct/2009:13:13:08 -0600] "HEAD /images/comprofiler/tnnull HTTP/1.1" 404 410 "5be9b90b912a4575cb0fcaf7de1d484f" "-"
74.208.198.241 - - [24/Oct/2009:13:13:09 -0600] "HEAD /images/comprofiler/tnnull HTTP/1.1" 404 410 "5be9b90b912a4575cb0fcaf7de1d484f" "-"
74.208.198.241 - - [24/Oct/2009:13:13:09 -0600] "HEAD /images/comprofiler/tnnull HTTP/1.1" 404 410 "5be9b90b912a4575cb0fcaf7de1d484f" "-"
74.208.198.241 - - [24/Oct/2009:13:13:09 -0600] "HEAD /images/comprofiler/tnnull HTTP/1.1" 404 410 "5be9b90b912a4575cb0fcaf7de1d484f" "-"
74.208.198.241 - - [24/Oct/2009:13:13:09 -0600] "HEAD /images/comprofiler/tnnull HTTP/1.1" 404 410 "5be9b90b912a4575cb0fcaf7de1d484f" "-"
74.208.198.241 - - [24/Oct/2009:13:13:10 -0600] "HEAD /images/comprofiler/tnnull HTTP/1.1" 404 410 "5be9b90b912a4575cb0fcaf7de1d484f" "-"
74.208.198.241 - - [24/Oct/2009:13:13:10 -0600] "HEAD /images/comprofiler/tnnull HTTP/1.1" 404 410 "5be9b90b912a4575cb0fcaf7de1d484f" "-"
74.208.198.241 - - [24/Oct/2009:13:13:10 -0600] "HEAD /images/comprofiler/tnnull HTTP/1.1" 404 410 "5be9b90b912a4575cb0fcaf7de1d484f" "-"

As you can see the site is getting the exact same call about 3-4 times per second from it's own IP.

nosing around, are you trying to put an avatar in the comments?
i searched the /images/comprofiler/tn part and that came up most

Well, that is just one site log and the URL is never consistent on the same site once I discover it and restart Apache.

Here is an excerpt from another log:

74.208.147.140 - - [24/Oct/2009:04:04:40 -0600] "HEAD /contact-us/ruminate-magazine.html HTTP/1.1" 404 373 "5be9b90b912a4575cb0fcaf7de1d484f" "-"
74.208.147.140 - - [24/Oct/2009:04:04:40 -0600] "HEAD /contact-us/ruminate-magazine.html HTTP/1.1" 404 373 "5be9b90b912a4575cb0fcaf7de1d484f" "-"
74.208.147.140 - - [24/Oct/2009:04:04:41 -0600] "HEAD /contact-us/ruminate-magazine.html HTTP/1.1" 404 373 "5be9b90b912a4575cb0fcaf7de1d484f" "-"
74.208.147.140 - - [24/Oct/2009:04:04:41 -0600] "HEAD /contact-us/ruminate-magazine.html HTTP/1.1" 404 373 "5be9b90b912a4575cb0fcaf7de1d484f" "-"
74.208.147.140 - - [24/Oct/2009:04:04:41 -0600] "HEAD /contact-us/ruminate-magazine.html HTTP/1.1" 404 373 "5be9b90b912a4575cb0fcaf7de1d484f" "-"
74.208.147.140 - - [24/Oct/2009:04:04:41 -0600] "HEAD /contact-us/ruminate-magazine.html HTTP/1.1" 404 373 "5be9b90b912a4575cb0fcaf7de1d484f" "-"
74.208.147.140 - - [24/Oct/2009:04:04:41 -0600] "HEAD /contact-us/ruminate-magazine.html HTTP/1.1" 404 373 "5be9b90b912a4575cb0fcaf7de1d484f" "-"
74.208.147.140 - - [24/Oct/2009:04:04:41 -0600] "HEAD /contact-us/ruminate-magazine.html HTTP/1.1" 404 373 "5be9b90b912a4575cb0fcaf7de1d484f" "-"
74.208.147.140 - - [24/Oct/2009:04:04:42 -0600] "HEAD /contests/short-story-contest.html HTTP/1.1" 404 373 "5be9b90b912a4575cb0fcaf7de1d484f" "-"
74.208.147.140 - - [24/Oct/2009:04:04:41 -0600] "HEAD /contact-us/ruminate-magazine.html HTTP/1.1" 404 373 "5be9b90b912a4575cb0fcaf7de1d484f" "-"
74.208.147.140 - - [24/Oct/2009:04:04:41 -0600] "HEAD /contact-us/ruminate-magazine.html HTTP/1.1" 404 373 "5be9b90b912a4575cb0fcaf7de1d484f" "-"
74.208.147.140 - - [24/Oct/2009:04:04:42 -0600] "HEAD /contact-us/ruminate-magazine.html HTTP/1.1" 404 373 "5be9b90b912a4575cb0fcaf7de1d484f" "-"
74.208.147.140 - - [24/Oct/2009:04:04:42 -0600] "HEAD /contact-us/ruminate-magazine.html HTTP/1.1" 404 373 "5be9b90b912a4575cb0fcaf7de1d484f" "-"
74.208.147.140 - - [24/Oct/2009:04:04:42 -0600] "HEAD /contact-us/ruminate-magazine.html HTTP/1.1" 404 373 "5be9b90b912a4575cb0fcaf7de1d484f" "-"
74.208.147.140 - - [24/Oct/2009:04:04:42 -0600] "HEAD /contests/short-story-contest.html HTTP/1.1" 404 373 "5be9b90b912a4575cb0fcaf7de1d484f" "-"
74.208.147.140 - - [24/Oct/2009:04:04:42 -0600] "HEAD /contact-us/ruminate-magazine.html HTTP/1.1" 404 373 "5be9b90b912a4575cb0fcaf7de1d484f" "-"
74.208.147.140 - - [24/Oct/2009:04:04:42 -0600] "HEAD /contact-us/ruminate-magazine.html HTTP/1.1" 404 373 "5be9b90b912a4575cb0fcaf7de1d484f" "-"
74.208.147.140 - - [24/Oct/2009:04:04:42 -0600] "HEAD /contact-us/ruminate-magazine.html HTTP/1.1" 404 373 "5be9b90b912a4575cb0fcaf7de1d484f" "-"
74.208.147.140 - - [24/Oct/2009:04:04:43 -0600] "HEAD /contact-us/ruminate-magazine.html HTTP/1.1" 404 373 "5be9b90b912a4575cb0fcaf7de1d484f" "-"
74.208.147.140 - - [24/Oct/2009:04:04:42 -0600] "HEAD /contact-us/ruminate-magazine.html HTTP/1.1" 404 373 "5be9b90b912a4575cb0fcaf7de1d484f" "-"

As you can see it is totally different. I have never seen anything like this before and after looking at the logs of all 3 sites it has just started happening in the last week. I can't figure it out why the site would start contacting itself. Also as you can see from the log file all of the requests are for pages that don't exist, hence the 404 on each line of the access.log

is there a bot trying to call anything before this loop?
its weird..

Not that I have found.
Another weird thing is the "HEAD" request in every line. Here is what apache says about "HEAD":

The HEAD method is identical to the GET method except that the server must not return a message-body in the response. The metainformation contained in the HTTP headers in response to a HEAD request should be identical to the information sent in response to a GET request. This allows a client to obtain meta-information about a resource without actually transferring the resource itself.

The head method is often used for testing hyperlinks, accessibility and for determining if a document has been recently modifed.

So why would the site be testing it's own hyperlinks? Makes me wonder if it is some sort of SQL injection.