How do I block Vertical Pigeon from my site?

General questions relating to Joomla! 1.5 There are other boards for more specific help on Joomla! features and extensions.

Moderator: General Support Moderators

Forum rules
Forum Rules
Absolute Beginner's Guide to Joomla! <-- please read before posting.
Forum Post Assistant - If you are serious about wanting help, you should use this tool to help you post.
Locked
kimberleyb
Joomla! Apprentice
Joomla! Apprentice
Posts: 7
Joined: Thu Jul 30, 2009 2:25 pm

How do I block Vertical Pigeon from my site?

Post by kimberleyb » Thu May 19, 2011 1:49 pm

TWO HUNDRED AND FIFTY FOUR 404 errors from these people.. THIS MONTH ALONE..

I've read the "reasons" for the rudeness and frankly, the peeping tom aspect of it and I don't buy it.

Anyone find a way yet?

Thanks.

User avatar
abernyte
Joomla! Virtuoso
Joomla! Virtuoso
Posts: 4076
Joined: Fri May 15, 2009 2:01 pm
Location: Écosse - Scozia - Escocia - Škotija -स्कॉटलैंड

Re: How do I block Vertical Pigeon from my site?

Post by abernyte » Thu May 19, 2011 5:45 pm

Short of blocking the referrer URLs from your logs via .htaccess and probably blocking tp=1 also via .htacess, and changing the Generator Tag which seems to be some of their search parameters, then...no. Neither blocks really disguise a Joomla site if you view the source code.

The above are not foolproof and site scrapers are a scourge we have to live with.
"Those who expect to reap the blessings of freedom must, like men, undergo the fatigue of supporting it." Thomas Paine

kimberleyb
Joomla! Apprentice
Joomla! Apprentice
Posts: 7
Joined: Thu Jul 30, 2009 2:25 pm

Re: How do I block Vertical Pigeon from my site?

Post by kimberleyb » Thu May 19, 2011 5:51 pm

thanks.. I appreciate the reply :)

User avatar
kenmcd
Joomla! Champion
Joomla! Champion
Posts: 5672
Joined: Thu Aug 18, 2005 2:09 am
Location: California
Contact:

Vertical Pigeon robots - HowTo Block with htaccess Rules

Post by kenmcd » Fri May 20, 2011 8:18 am

.
Below are htaccess file rules to block the intrusive Vertical Pigeon robots from scanning your Joomla website.

Information found in server logs about Vertical Pigeon information gathering robots

User agent:
Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; http://[URL banned].com/)

Referrer:
http://[URL banned].com/

Robot IP addresses:
66.103.57.22
66.103.61.161
67.58.209.11
(if you find any more IP addresses in your logs please post them here)


The most efficient way to block them is to block the IP addresses where their robots are coming from.
This method is probably the lowest load on your server.

Blocking Vertical Pigeon robots by IP Address
This method should be at the top of the .htaccess file.

Code: Select all

## Block Vertical Pigeon robots by IP Address
Order allow,deny
Deny from 66.103.57.22
Deny from 66.103.61.161
Deny from 67.58.209.11
Allow from all
The following methods should be after the RewriteEngine On line but before other rules.
If the IP address blocks above are working, these methods are not needed.

Blocking Vertical Pigeon robots by Referrer

Code: Select all

## Block Vertical Pigeon robots by Referrer
RewriteCond %{HTTP_REFERER} [URL banned]\.com [NC]
RewriteRule . - [F]
or to send their robots back to their own website

Code: Select all

## Block Vertical Pigeon robots by Referrer 
RewriteCond %{HTTP_REFERER} [URL banned]\.com [NC]
RewriteRule . http://[URL banned].com/scumbags.html [L]
You can use what ever page name you want.
The scumbags.html is just an example 404 entry for them to read in their website logs.


Blocking Vertical Pigeon robots by User Agent

Code: Select all

## Block Vertical Pigeon robots by User Agent
RewriteCond %{HTTP_USER_AGENT} [URL banned] [NC]
RewriteRule . - [F]
All of these have been tested locally and they do not cause any immediate server errors.
So you will have to test on your own servers to see if the rules are actually working as expected.


Note that I was going to post this in the other Vertical Pigeon thread to keep all the info together,
but that thread has been locked.
http://forum.joomla.org/viewtopic.php?f=432&t=598785
.
██ LibreTraining

User avatar
mandville
Joomla! Master
Joomla! Master
Posts: 15098
Joined: Mon Mar 20, 2006 1:56 am
Location: The Girly Side of Joomla in Sussex

Re: How do I block Vertical Pigeon from my site?

Post by mandville » Mon Jul 18, 2011 7:00 am

an update on the VP
here are some samples of the pages called for when their bot hits.



/templates/template/templateDetails.xml
/robots.txt
/plugins/editors/tinymce.xml
/index2.php?option=com_contact
/index.php?option=com_qjnljhlbttyvdnjl
/htaccess.txt
/configuration.php-dist
/components/com_contact/views/contact/tmpl/default.xml
/?tp=1
/?format=feed
wonder what this test site would sho on their system

Code: Select all

/** @var int Main Release Level */
	var $RELEASE 	= '5'.1;
	/** @var string Development Status */
	var $DEV_STATUS = 'Stable';
	/** @var int Sub Release Level */
	var $DEV_LEVEL 	= 'vp';
HU2HY- Poor questions = Poor answer
Un requested Help PM's will be reported, added to the foe list and possibly just deleted
{VEL Team Leader}{TM Auditor }{ Showcase & Security forums Moderator}

User avatar
kenmcd
Joomla! Champion
Joomla! Champion
Posts: 5672
Joined: Thu Aug 18, 2005 2:09 am
Location: California
Contact:

Re: How do I block Vertical Pigeon from my site?

Post by kenmcd » Tue Jul 19, 2011 11:42 am

.
Interesting.
Wish more people would post what they are seeing. :)
Seen any other IP addresses for the bot?

Most of those look like Joomla website fingerprinting.

The XML files for the template and extensions are them cataloging what is used on the site and versions.

The call to the com_contact page is probably them scraping contact information.


Looks like the best way to deal with Vertical Pidgeon is still to block their bots.
.
██ LibreTraining

User avatar
mandville
Joomla! Master
Joomla! Master
Posts: 15098
Joined: Mon Mar 20, 2006 1:56 am
Location: The Girly Side of Joomla in Sussex

Re: How do I block Vertical Pigeon from my site?

Post by mandville » Tue Jul 19, 2011 12:01 pm

forgot to add, it was known IP for them,. and they ignored the robot file on this "special" site
HU2HY- Poor questions = Poor answer
Un requested Help PM's will be reported, added to the foe list and possibly just deleted
{VEL Team Leader}{TM Auditor }{ Showcase & Security forums Moderator}

User avatar
kenmcd
Joomla! Champion
Joomla! Champion
Posts: 5672
Joined: Thu Aug 18, 2005 2:09 am
Location: California
Contact:

Block Vertical Pigeon with EasyCalcCheck PLUS plug-in

Post by kenmcd » Tue Jul 19, 2011 12:27 pm

.
Hmmmm . . . if they ignore the robots.txt file the Bot-trap script should also work.
Bot-trap - A Bad Web-Robot Blocker - automatically ban bad web robots (aka web spiders) that ignore the robots.txt file.

The EasyCalcCheck PLUS plug-in includes Bot-trap.
So that is another option for users to block Vertical Pigeon-droppings.

EasyCalcCheck PLUS (P)
http://extensions.joomla.org/extensions ... tcha/11964
ReCaptcha, Akismet, Bot-Trap, Honeypot Project, Mollom, StopForumSpam, BotScout

.
██ LibreTraining

graemelg
Joomla! Apprentice
Joomla! Apprentice
Posts: 24
Joined: Wed Dec 05, 2007 2:43 pm

Re: How do I block Vertical Pigeon from my site?

Post by graemelg » Wed Jul 20, 2011 7:00 pm

Should I be concerned that [URL banned].com shows up as the referrering page for about 160 entries in the Redirect Manager of my J1.7 site? There is no New URL specified for any of them, so should I assume this is just an artifact of their scraping?

vicoba
Joomla! Fledgling
Joomla! Fledgling
Posts: 2
Joined: Sat Dec 10, 2011 10:50 am

Re: How do I block Vertical Pigeon from my site?

Post by vicoba » Sat Dec 10, 2011 11:06 am

Hi to all!
As you can see, this is my first post :) although i'm reading a lot of great stuff here... :)
My question is: did anyone successfully blocked Vertical Pigeon from accessing to site?
I've tried everything including advises from this and other parts of forum that are concerning this matter and did not have any success.
Any help is appreciated and thank You in advance...

balams
Joomla! Apprentice
Joomla! Apprentice
Posts: 12
Joined: Mon Apr 04, 2011 9:05 pm

Re: How do I block Vertical Pigeon from my site?

Post by balams » Thu Jan 12, 2012 2:32 pm

why is Vp snooping around
I have more than 300 entries from vp.
it looks like their software bots are indexing the web for joomla sites
what is the reason? / their website has not explainations of why?
they said that is a joomla tool to find out your joomla install software.
but any one administrator can do this by going to the admin area.
no need of a tool like this,
is Vp a backdoor site search?
what do you guys think about Vp

User avatar
mandville
Joomla! Master
Joomla! Master
Posts: 15098
Joined: Mon Mar 20, 2006 1:56 am
Location: The Girly Side of Joomla in Sussex

Re: How do I block Vertical Pigeon from my site?

Post by mandville » Thu Jan 12, 2012 4:05 pm

mandville wrote:please use the forum search for vertical pigeon to see the options available.
sadly not all countries share the same legal opinion
HU2HY- Poor questions = Poor answer
Un requested Help PM's will be reported, added to the foe list and possibly just deleted
{VEL Team Leader}{TM Auditor }{ Showcase & Security forums Moderator}

User avatar
oldskule
Joomla! Apprentice
Joomla! Apprentice
Posts: 14
Joined: Fri Sep 02, 2005 6:18 pm
Location: Midwest, USA
Contact:

Re: How do I block Vertical Pigeon from my site?

Post by oldskule » Fri Aug 17, 2012 12:26 am

Here's 3 more IP's that weren't listed in the thread.

66.103.62.11
66.206.230.200
67.22.142.16

Code: Select all

66.103.62.11 - - [18/Apr/2012:15:45:58 -0400] "GET /robots.txt HTTP/1.0" 404 298 "-" "Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0)"
66.103.62.11 - - [18/Apr/2012:15:45:58 -0400] "GET /?tp=1 HTTP/1.0" 200 7739 "http://[URL banned].com/" "Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0)"

66.206.230.200 - - [20/Jan/2012:07:00:03 -0500] "GET /?tp=1 HTTP/1.0" 200 107018 "http://[URL banned].com/" "Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0)"
66.206.230.200 - - [20/Jan/2012:07:00:04 -0500] "GET /configuration.php-dist HTTP/1.0" 200 3411 "http://[URL banned].com/" "Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0)"
66.206.230.200 - - [20/Jan/2012:07:00:04 -0500] "GET /htaccess.txt HTTP/1.0" 200 3596 "http://[URL banned].com/" "Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0)"

67.22.142.16 - - [06/Aug/2012:15:58:10 +0000] "GET /?tp=1 HTTP/1.0" 200 70209 "http://[URL banned].com/" "Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0)"
67.22.142.16 - - [06/Aug/2012:15:58:11 +0000] "GET /configuration.php-dist HTTP/1.0" 200 3411 "-" "Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0)"
67.22.142.16 - - [06/Aug/2012:15:58:11 +0000] "GET /htaccess.txt HTTP/1.0" 200 2815 "-" "Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0)"

michaeldobritz
Joomla! Apprentice
Joomla! Apprentice
Posts: 32
Joined: Fri Jul 02, 2010 2:36 am

Re: How do I block Vertical Pigeon from my site?

Post by michaeldobritz » Mon Nov 05, 2012 1:18 pm

Just had the pigeon hit me the first time. Can somebody just post again what exactly has to be added to the .htaccess to stop that VP robot? Please!

User avatar
PhilD
Joomla! Hero
Joomla! Hero
Posts: 2734
Joined: Sat Oct 21, 2006 10:20 pm
Location: Wisconsin USA
Contact:

Re: How do I block Vertical Pigeon from my site?

Post by PhilD » Tue Nov 06, 2012 4:28 am

The info posted above by kenmcd is what you want. However it may not always prevent the robots as they are determined to scan sites and use a variety of tactics.
PhilD -- Unrequested PM's and/or emails may not get a response.
Security Moderator

User avatar
mandville
Joomla! Master
Joomla! Master
Posts: 15098
Joined: Mon Mar 20, 2006 1:56 am
Location: The Girly Side of Joomla in Sussex

Re: How do I block Vertical Pigeon from my site?

Post by mandville » Wed May 08, 2013 1:36 pm

intersting to read what they collected despite assurances they didnt
The Data

detailed information on 2,791,291 Joomla!/Mambo websites throughout the world, including:
complete DNS info
complete WHOIS info
complete server info
DMOZ info
CMS version
SSL information
date CMS installed
scanned for 156 common Joomla! extensions, with versions and installation dates
limited information on a further 17,000,000 non-Joomla websites
Contact info:
Joomla! administrators and users names and email addresses
website contact name, address, city, ZIP/postal, country, phone, mobile, fax, latitude, longitude, etc.


i have removed the standard get anywhere information and left the more disturbing stuff
HU2HY- Poor questions = Poor answer
Un requested Help PM's will be reported, added to the foe list and possibly just deleted
{VEL Team Leader}{TM Auditor }{ Showcase & Security forums Moderator}

User avatar
oldskule
Joomla! Apprentice
Joomla! Apprentice
Posts: 14
Joined: Fri Sep 02, 2005 6:18 pm
Location: Midwest, USA
Contact:

Re: How do I block Vertical Pigeon from my site?

Post by oldskule » Wed May 08, 2013 1:44 pm

Very interesting for sure.

It's this line that creeps me out a bit.
Joomla! administrators and users names and email addresses
Let's hope their massive database goes to someone who doesn't intend to use it for some dishonest purpose. I'm glad we've been blocking them for most of the 4 years they were scraping.

handy andy
Joomla! Enthusiast
Joomla! Enthusiast
Posts: 109
Joined: Fri Aug 06, 2010 1:56 pm

Re: How do I block Vertical Pigeon from my site?

Post by handy andy » Fri Sep 27, 2013 8:26 pm

Hi,
3 web sites hacked and injected with a dangerous form of malware, all three have one thing in common, Vertical Pigeon.com

The sites have been blacklisted by Google and must be rebuilt from the start, new databases, FTP settings, Joomla uploads, then to try and save the content and more important the existing links or as happened before the new sites will be throwing up 404 errors for weeks.

Google were very fast to blacklist the sites rightly so due to the nature of the malware, however the hackers who appear to have been assisted by Vertical Pigeon.com are allowed to continue.

No justice and a definite reason to begin looking for another C.M.S. so small business can use the internet.

User avatar
mandville
Joomla! Master
Joomla! Master
Posts: 15098
Joined: Mon Mar 20, 2006 1:56 am
Location: The Girly Side of Joomla in Sussex

Re: How do I block Vertical Pigeon from my site?

Post by mandville » Sat Sep 28, 2013 1:33 am

can you be sure that VP actually hacked your site, or did someone scan your site for weakness using VP?
run and post the fpa in a security topic so it can be checked
HU2HY- Poor questions = Poor answer
Un requested Help PM's will be reported, added to the foe list and possibly just deleted
{VEL Team Leader}{TM Auditor }{ Showcase & Security forums Moderator}

handy andy
Joomla! Enthusiast
Joomla! Enthusiast
Posts: 109
Joined: Fri Aug 06, 2010 1:56 pm

Re: How do I block Vertical Pigeon from my site?

Post by handy andy » Sat Sep 28, 2013 1:22 pm

Hi mandville,

I know I am working on an assumption based on the coincidence that vertical pigeon were in the site the same day.

The malware infection ran very deep on two of the sites, so deep the only way to get them cleaned and the Google blacklist removed was to delete literally everything stopping short at cancelling the domain registration.

Both sites were running Joomla 2.5.14, no unnecessary plug ins or components not even JCE.

There was nothing personal or any big money to be made from the hack because the sites are designed for local use not world wide as they belong to tradesmen who are targeting the specific local area where their businesses are located.

I didn't have time to run the fpa as the sites are the only advertising (besides referrals) these people use to get business.

Google have been very quick to remove the blacklist status, hopefully it will not take too long to get their good ranking back.

If vertical puigeon did not actually hack the sites they certainly had the access as there were a large number of re-directs to them viewed in the database, the type of information these people appear to claim to sell.

I am putting in a firewall by DMC http://extensions.joomla.org/extensions ... tion/23659 in the hope that it may help against future attacks.

Regards,

Andy.

lara_19800
Joomla! Apprentice
Joomla! Apprentice
Posts: 11
Joined: Sun Sep 22, 2013 10:22 am

Re: How do I block Vertical Pigeon from my site?

Post by lara_19800 » Thu Oct 03, 2013 10:00 pm

oldskule wrote:Here's 3 more IP's that weren't listed in the thread.

66.103.62.11
66.206.230.200
67.22.142.16

Code: Select all

66.103.62.11 - - [18/Apr/2012:15:45:58 -0400] "GET /robots.txt HTTP/1.0" 404 298 "-" "Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0)"
66.103.62.11 - - [18/Apr/2012:15:45:58 -0400] "GET /?tp=1 HTTP/1.0" 200 7739 "http://[URL banned].com/" "Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0)"

66.206.230.200 - - [20/Jan/2012:07:00:03 -0500] "GET /?tp=1 HTTP/1.0" 200 107018 "http://[URL banned].com/" "Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0)"
66.206.230.200 - - [20/Jan/2012:07:00:04 -0500] "GET /configuration.php-dist HTTP/1.0" 200 3411 "http://[URL banned].com/" "Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0)"
66.206.230.200 - - [20/Jan/2012:07:00:04 -0500] "GET /htaccess.txt HTTP/1.0" 200 3596 "http://[URL banned].com/" "Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0)"

67.22.142.16 - - [06/Aug/2012:15:58:10 +0000] "GET /?tp=1 HTTP/1.0" 200 70209 "http://[URL banned].com/" "Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0)"
67.22.142.16 - - [06/Aug/2012:15:58:11 +0000] "GET /configuration.php-dist HTTP/1.0" 200 3411 "-" "Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0)"
67.22.142.16 - - [06/Aug/2012:15:58:11 +0000] "GET /htaccess.txt HTTP/1.0" 200 2815 "-" "Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0)"
As you can see, this is my first post :) although i'm reading a lot of great stuff here... :)
My question is: did anyone successfully blocked Vertical Pigeon from accessing to site?
I've tried everything including advises from this and other parts of forum that are concerning this matter and did not have any success.
Any help is appreciated and thank You in advance...
Please read the Forum rules regarding signatures: http://forum.joomla.org/viewtopic.php?t=65


Locked

Return to “General Questions/New to Joomla! 1.5”