[ABD] Bot Indexing MOD v1.1.1

A place for MOD Authors to post and receive feedback on MODs still in development. No MODs within this forum should be used within a live environment! No new topics are allowed in this forum.
Forum rules
READ: phpBB.com Board-Wide Rules and Regulations

IMPORTANT: MOD Development Forum rules

On February 1, 2009 this forum will be set to read only as part of retiring of phpBB2.
Kemars
Registered User
Posts: 38
Joined: Wed Jan 07, 2004 6:57 am
Location: Helsinki, Finland
Contact:

Post by Kemars » Sun Oct 30, 2005 3:25 pm

Flash005 wrote:
Flash005 wrote:Instead of having like googlebot (3) i have - googlebot, googlebot , googlebot

Can someone help me on this one?


Let me try again..

Does anyone knows how to solve this?


Follow the steps below exactly as shown in their mods respectfully, and it should work perfectly..

------------------------------------------------------------------------------
http://www.phpbb.com/phpBB/viewtopic.php?t=200361 - for bot indexing mod - v1.2 i do believe.


then add this code below

Code: Select all


# 
#-----[ OPEN ]---------------------------------------- 
# 
includes/page_header.php 

# 
#-----[ FIND ]----------------------------------- 
# 
            if( $row['is_robot'] ) 
            { 
               $online_userlist = $row['is_robot'] . (($online_userlist!='') ? ', ' : '') . $online_userlist; 
            } 

# 
#-----[ REPLACE WITH ]----------------------------- 
# 
            if( $row['is_robot'] ) 
            { 
               $RbList = ''; 
               if($row['is_robot']!="0") // If it's not zero it can only be a Bot's Name 
               { 
                  $BotName = $row['is_robot']; // Extract is_robot's datas into a dedicaced array 
                  if (!isset($arrRobot[$BotName])) 
                  { 
                     $arrRobot[$BotName]=1; // If first Bot Name's view, define 1 
                  } 
                  else 
                  { 
                     $arrRobot[$BotName]++; // If it's viewed again, just $var++ 
                  } 
               } 
            } 

# 
#-----[ FIND ]----------------------------------- 
# 
   if ( empty($online_userlist) ) 
   { 
      $online_userlist = $lang['None']; 
   } 

# 
#-----[ INSERT BEFORE ]----------------------------- 
# 
   if(is_array($arrRobot)) 
   { 
      foreach($arrRobot as $RbName=>$nbOnline) 
      { 
         $RbList .= $RbName . (($nbOnline > 1) ? "($nbOnline), " : ", "); 
      } 
   } 
   $online_userlist = "<i>".$RbList."</i>" . $online_userlist; 

# 
#-----[ SAVE/CLOSE ALL FILES ]------------------------------------------ 
# 
# EoM  
Now in the Admin Control Panel (ACP) you can also do a few changes so that it shows the actual name of the bot rather than just guest -- you can now find out who really is a guest and which bot is indexing your site..

try this one - http://www.phpbb.com/phpBB/viewtopic.ph ... 87#1732587
How much wood would a woodchuck chuck if a woodchuck could chuck wood?
http://www.iesaf.fi / http://www.kemars.com

TPFKAS
Registered User
Posts: 87
Joined: Thu Nov 18, 2004 10:59 pm

Post by TPFKAS » Sun Oct 30, 2005 7:52 pm

Hi guys,
Maybe this question has come up before in this (80 page!) thread, so apologies in advance for repeating it.
I've got this mod on my forum and I see the the bots are visiting the site. Since a couple of weeks I see that almost every week the "pending bots" section contains a whole list of differnet IP-adresses which all are assigned to Googlebot. I keep adding these addresses, but I am beginning to wonder if these are really Googlebot visits or can the system be fooled by other visits who pretend to be bots?

AuctionMan
Registered User
Posts: 28
Joined: Mon Aug 01, 2005 11:37 pm
Contact:

Post by AuctionMan » Tue Nov 01, 2005 12:02 am

Will updating to ver2.0.18 affect this mod now I have it installed and running properly I wouldn't want to ruin the hard work!!

Mike
dropshipping article and sources.
Credit Cards - use the 0% deals to make money.
ebay templates - free to use plus great sources for more templates
Ebay Forum ebay to online income the easy and sustainable way.

ridinhighspeeds
Registered User
Posts: 105
Joined: Tue Jan 25, 2005 1:53 am
Location: MA
Contact:

Post by ridinhighspeeds » Tue Nov 01, 2005 5:15 am

What ever happened to the creater of the MOD? He/She did a great job creating this MOD...but their are a lot of cool features many people have added in addition to the MOD that should be released in a new BETA.

kulinar
Registered User
Posts: 651
Joined: Thu Feb 17, 2005 7:31 pm
Contact:

Post by kulinar » Tue Nov 01, 2005 7:34 am

AuctionMan wrote: Will updating to ver2.0.18 affect this mod now I have it installed and running properly I wouldn't want to ruin the hard work!!

Mike
Works fine on 2.0.18 so far (2 days) :wink:.

baerenwurm
Registered User
Posts: 131
Joined: Wed Nov 06, 2002 2:38 pm
Location: Germany

Post by baerenwurm » Wed Nov 02, 2005 7:06 pm

sry, double post. :oops:
Last edited by baerenwurm on Wed Nov 02, 2005 7:08 pm, edited 1 time in total.

baerenwurm
Registered User
Posts: 131
Joined: Wed Nov 06, 2002 2:38 pm
Location: Germany

Post by baerenwurm » Wed Nov 02, 2005 7:07 pm

A few IPs of Inktomi aren't recognized such as 68.142.249.180 although I have specified "Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp)" etc. in "Agent match".

2ct7
Registered User
Posts: 19
Joined: Tue May 31, 2005 6:13 pm

Post by 2ct7 » Thu Nov 03, 2005 9:24 pm

Can this mod reduce the bandwidth that the googlebot eats up?

User avatar
EverettB
Registered User
Posts: 326
Joined: Fri Aug 01, 2003 7:11 pm
Location: North America

Post by EverettB » Thu Nov 03, 2005 9:32 pm

TPFKAS wrote: I've got this mod on my forum and I see the the bots are visiting the site. Since a couple of weeks I see that almost every week the "pending bots" section contains a whole list of differnet IP-adresses which all are assigned to Googlebot. I keep adding these addresses, but I am beginning to wonder if these are really Googlebot visits or can the system be fooled by other visits who pretend to be bots?

They are probably fake. I wouldn't add them unless you check the IP address first and make sure it is Google.
Last edited by EverettB on Thu Nov 03, 2005 9:47 pm, edited 1 time in total.

User avatar
EverettB
Registered User
Posts: 326
Joined: Fri Aug 01, 2003 7:11 pm
Location: North America

Post by EverettB » Thu Nov 03, 2005 9:33 pm

2ct7 wrote: Can this mod reduce the bandwidth that the googlebot eats up?

It increases the bandwidth Google uses by allowing it to travel through more of your site.

2ct7
Registered User
Posts: 19
Joined: Tue May 31, 2005 6:13 pm

Post by 2ct7 » Thu Nov 03, 2005 9:43 pm

I don't want to log googlebot out but it uses so much bandwidth my provider is complaining. Is there a Mod that appeases the googlebot but somehow reduces its bandwidth gobbling?

User avatar
EverettB
Registered User
Posts: 326
Joined: Fri Aug 01, 2003 7:11 pm
Location: North America

Post by EverettB » Thu Nov 03, 2005 9:58 pm

You can use a robots.txt file to block Googlebot from completely indexing your site, if you want.

Here's one robots.txt tutorial I found:
http://www.searchengineworld.com/robots ... torial.htm

I'm not sure how to just reduce it, although I have my robots.txt blocking certain pages that I don't want indexed, which will reduce it somewhat.
Sample:

Code: Select all

User-agent: *
Disallow: /forum/admin/
Disallow: /forum/db/
Disallow: /forum/images/
Disallow: /forum/includes/
Disallow: /forum/language/
Disallow: /forum/templates/
Disallow: /forum/common.php
Disallow: /forum/config.php
Disallow: /forum/faq.php
Disallow: /forum/groupcp.php
Disallow: /forum/login.php
Disallow: /forum/memberlist.php
Disallow: /forum/modcp.php
Disallow: /forum/posting.php
Disallow: /forum/privmsg.php
Disallow: /forum/profile.php
Disallow: /forum/search.php
Disallow: /forum/viewonline.php

TPFKAS
Registered User
Posts: 87
Joined: Thu Nov 18, 2004 10:59 pm

Post by TPFKAS » Thu Nov 03, 2005 10:00 pm

EverettB wrote: They are probably fake. I wouldn't add them unless you check the IP address first and make sure it is Google.


So how do I check those IP addresses?
I found several places on the web that list IP's fopr Googlebot, but they don't seem to agree and are not always up-to-date.

Thanks

2ct7
Registered User
Posts: 19
Joined: Tue May 31, 2005 6:13 pm

Post by 2ct7 » Thu Nov 03, 2005 10:08 pm

Actually I don't want to ban google, I just want to stop it endlessly idexing my forums and eating gigs and gigs of bandwidth.

This thread http://www.phpbb.com/phpBB/viewtopic.ph ... sc&start=0 looks sort of like it does it but it does not appear to be an approved MOD and the sessions.php they want me to edit looks nothing like the current one.

Is there no MOD to reduce Google's bandwidth usage? This is a really serious problem that could cripple any phpbb board.

2ct7
Registered User
Posts: 19
Joined: Tue May 31, 2005 6:13 pm

Post by 2ct7 » Thu Nov 03, 2005 10:15 pm

TPFKAS - did you do a reverse IP look up? You can search google for one. The ip flooding my forum is registered to google.

EverettB - I'll the robot.txt and see if that does anything thanks!

Locked

Return to “[2.0.x] MODs in Development”