[INFO] How gogle PHPBB!

[mushy]

Ok, I got it. Thanks... I will try it and see the difference then...


But I have one more question. You wrote user agent *


Does * means all the spiders that crawl or just google? I am having problem with inkomi using multiple sessions also

[Kharay]

User-agent: * means every bot/browser/machine/thing that reads and acknowledges robots.txt

[raintree]

hi,

I'm new to here. I'm using 2.06, I read some of the thread, by chance, which mod has no problem so far???

Can anyone care to share it here?

thanks in advance.

[Nicola]

There is another problem with this Mod.

################################
## MOD Title: Static-looking phpBB pages
## MOD Author: Acid Paul
################################

I install. it and it works, but if i have a new post in a Topic with 2 or more Sites and i push this button:
It don’t will take you to the new post on the 2 or more site. Because the URL ist so:
ftopic1809.html&view=newest
but it must be like
ftopic1809-15.html&view=newest

There is something wrong.


Sorry about my terrible English.

[Pablasso]

I have modified the Googlifier mod for PHPNuke with PHPBB2 port found at www.orientalgate.org to work with standalone PHPBB2. This mod allows you to turn dynamic forum urls into static ones and therefore get better indexed and ranked by Google & co., as well as picked by search engines, that don't like dynamic URLs (e.g. AllTheWeb).

The effect is like this:

> viewtopic.php?t=157 => ftopic28.html

(affects all topics, posts, forum folders and pagination; member profiles, search, lorgin and usergroups URL stay unchanged; the main page remains index.php)

Requirements: you need mod_rewrite enabled as well as ability to use .htaccess or modify apache config files.

Implementation:

Step 1. In /includes/page_header.php before

Code: Select all

//
// Generate logged in/logged out status
//

add this code (make sure there are no space breaks at line ends after you paste):

Code: Select all

ob_start();
function replace_for_mod_rewrite(&$s)
{
$urlin =
array(
"'(?<!/)viewforum.php\?f=([0-9]*)&topicdays=([0-9]*)&start=([0-9]*)'",
"'(?<!/)viewforum.php\?f=([0-9]*)&mark=topics'",
"'(?<!/)viewforum.php\?f=([0-9]*)'",
"'(?<!/)viewtopic.php\?t=([0-9]*)&view=previous'",
"'(?<!/)viewtopic.php\?t=([0-9]*)&view=next'",
"'(?<!/)viewtopic.php\?t=([0-9]*)&postdays=([0-9]*)&postorder=([a-zA-Z]*)&start=([0-9]*)'",
"'(?<!/)viewtopic.php\?t=([0-9]*)&start=([0-9]*)&postdays=([0-9]*)&postorder=([a-zA-Z]*)&highlight=([a-zA-Z0-9]*)'",
"'(?<!/)viewtopic.php\?t=([0-9]*)&start=([0-9]*)'",
"'(?<!/)viewtopic.php\?t=([0-9]*)'",
"'(?<!/)viewtopic.php&p=([0-9]*)'",
"'(?<!/)viewtopic.php\?p=([0-9]*)'",
);
$urlout = array(
"viewforum\\1-\\2-\\3.html",
"forum\\1.html",
"forum\\1.html",
"ptopic\\1.html",
"ntopic\\1.html",
"ftopic\\1-\\2-\\3-\\4.html",
"ftopic\\1.html",
"ftopic\\1-\\2.html",
"ftopic\\1.html",
"sutra\\1.html",
"sutra\\1.html",
);
$s = preg_replace($urlin, $urlout, $s);
return $s;
}

Step 2. In /includes/page_tail.php after

Code: Select all

$db->sql_close();

add this:

Code: Select all

$contents = ob_get_contents();
ob_end_clean();
echo replace_for_mod_rewrite($contents);
global $dbg_starttime;


in the same file after

Code: Select all

ob_end_clean();

add this:

Code: Select all

echo replace_for_mod_rewrite($contents);
global $dbg_starttime;

Step 3. In your .htaccess file (if you don't have one create it; should be located in you forum root directory) paste these lines:

Code: Select all

RewriteEngine On
RewriteRule ^forums.* index.php
RewriteRule ^forum([0-9]*).* viewforum.php?f=$1&mark=topic
RewriteRule ^viewforum([0-9]*)-([0-9]*)-([0-9]*).* viewforum.php?f=$1&topicdays=$2&start=$3
RewriteRule ^forum([0-9]*).* viewforum.php?f=$1
RewriteRule ^ptopic([0-9]*).* viewtopic.php?t=$1&view=previous
RewriteRule ^ntopic([0-9]*).* viewtopic.php?t=$1&view=next
RewriteRule ^ftopic([0-9]*)-([0-9]*)-([a-zA-Z]*)-([0-9]*).* viewtopic.php?t=$1&postdays=$2&postorder=$3&start=$4
RewriteRule ^ftopic([0-9]*)-([0-9]*).* viewtopic.php?t=$1&start=$2
RewriteRule ^ftopic([0-9]*).* viewtopic.php?t=$1
RewriteRule ^ftopic([0-9]*).html viewtopic.php?t=$1&start=$2&postdays=$3&postorder=$4&highlight=$5
RewriteRule ^sutra([0-9]*).* viewtopic.php?p=$1

Note: sometimes this .htaccess code won't work properly if the forum is on a subdomain (was the case with me), you may need to try this variation:

Code: Select all

RewriteEngine On
RewriteRule ^forums.* /index.php
RewriteRule ^forum([0-9]*).* /viewforum.php?f=$1&mark=topic
RewriteRule ^viewforum([0-9]*)-([0-9]*)-([0-9]*).* /viewforum.php?f=$1&topicdays=$2&start=$3
RewriteRule ^forum([0-9]*).* /viewforum.php?f=$1
RewriteRule ^ptopic([0-9]*).* /viewtopic.php?t=$1&view=previous
RewriteRule ^ntopic([0-9]*).* /viewtopic.php?t=$1&view=next
RewriteRule ^ftopic([0-9]*)-([0-9]*)-([a-zA-Z]*)-([0-9]*).* /viewtopic.php?t=$1&postdays=$2&postorder=$3&start=$4
RewriteRule ^ftopic([0-9]*)-([0-9]*).* /viewtopic.php?t=$1&start=$2
RewriteRule ^ftopic([0-9]*).* /viewtopic.php?t=$1
RewriteRule ^ftopic([0-9]*).html /viewtopic.php?t=$1&start=$2&postdays=$3&postorder=$4&highlight=$5
RewriteRule ^sutra([0-9]*).* /viewtopic.php?p=$1

Step 4. Very important! In your robots.txt file (goes at the *site* root) add these lines:

Code: Select all

Disallow: /your-forum-folder/sutra*.html$
Disallow: /your-forum-folder/ptopic*.html$
Disallow: /your-forum-folder/ntopic*.html$
Disallow: /your-forum-folder/ftopic*asc*.html$

(This is required to avoid feeding duplicate content to Google)

Step 5. Apply Google Mod #1 described at this page.

Congrats! Now you now have traffic-generating mogul.

im trying to apply this... all is good and the mod works well (excellent acid!) but i found that with posts with 2 or more pages simply doesnt work... when i tried to go to page #2 or other than #1, it always send me to page #1

how can i solve this?

another thing... i have to put this line in .htaccess:

Options +followsymlinks

cuz without it mod_rewrite doesnt works on my host... this affects me?

thanks!

[mr.luc]

a little question..
i'm using the basic mod on 2.06.. and google comes nearly daily..

but now a user "warned" me that the profiles were in google also..

i read somewhere in this thread that people exclude the profiles from spidering..

i did not include them in the robots.txt... on purpose..
because on my site each profile is like a little homepage.. with guestbook and pictures.. and so on.. and i thought that users were happy to be listed in google..
so here is my question:
is there another reason than a little paranoia from some users, to exclude the profiles from being spidered by google???

(i have enough bandwith and the server is trong enough)

Luc

[larsneo]

but i found that with posts with 2 or more pages simply doesnt work... when i tried to go to page #2 or other than #1, it always send me to page #1

this might happen if using (HTML compliant) & instead of & in your links...

Step 1. In /includes/page_header.php before

Code: Select all

//
// Generate logged in/logged out status
//

add this code (make sure there are no space breaks at line ends after you paste):

Code: Select all

//
// Short URL implementation
//
ob_start();
function replace_for_mod_rewrite(&$s)
{
$urlin =
array(
"'(?<!/)viewforum.php\?f=([0-9]*)&(?:amp;)topicdays=([0-9]*)&(?:amp;)start=([0-9]*)'",
"'(?<!/)viewforum.php\?f=([0-9]*)&(?:amp;)mark=topics'",
"'(?<!/)viewforum.php\?f=([0-9]*)'",
"'(?<!/)viewtopic.php\?t=([0-9]*)&(?:amp;)view=previous'",
"'(?<!/)viewtopic.php\?t=([0-9]*)&(?:amp;)view=next'",
"'(?<!/)viewtopic.php\?t=([0-9]*)&(?:amp;)postdays=([0-9]*)&(?:amp;)postorder=([a-zA-Z]*)&(?:amp;)start=([0-9]*)'",
"'(?<!/)viewtopic.php\?t=([0-9]*)&(?:amp;)start=([0-9]*)&(?:amp;)postdays=([0-9]*)&(?:amp;)postorder=([a-zA-Z]*)&(?:amp;)highlight=([a-zA-Z0-9]*)'",
"'(?<!/)viewtopic.php\?t=([0-9]*)&(?:amp;)start=([0-9]*)'",
"'(?<!/)viewtopic.php\?t=([0-9]*)'",
"'(?<!/)viewtopic.php&(?:amp;)p=([0-9]*)'",
"'(?<!/)viewtopic.php\?p=([0-9]*)'",
);
$urlout = array(
"viewforum\\1-\\2-\\3.html",
"forum\\1.html",
"forum\\1.html",
"ptopic\\1.html",
"ntopic\\1.html",
"ftopic\\1-\\2-\\3-\\4.html",
"ftopic\\1.html",
"ftopic\\1-\\2.html",
"ftopic\\1.html",
"sutra\\1.html",
"sutra\\1.html",
);
$s = preg_replace($urlin, $urlout, $s);
return $s;
}

[gurukast]

My session.php has only included the mod which will enhance spider by the google. But I didn't include the mod which prevent google to use multiple session since I got confused where exactly to put those codes...... I am not getting confused in the sense. Look at the mod, only one part is included but not the second. If you can help me with the second part (to prevent multiple session by the google) then it would be a big help....

Don't do that "prevent multiple sessions for Google" MOD. Trust me, you don't need it. And it will only break the whole thing apart.

'


actually i do need it.. i can't afford googlebot to be logginf in twenty times and creating a new session id every time... the table would get swamped

so this code...

Code: Select all

{
      $sessiondata = array();
      $session_id = ( isset($HTTP_GET_VARS['sid']) ) ? $HTTP_GET_VARS['sid'] : '';
      $sessionmethod = SESSION_METHOD_GET;
   }

should i just add this after all instances?

Code: Select all

   if ( empty($session_id)  && strstr($HTTP_SERVER_VARS['HTTP_USER_AGENT'] ,'Googlebot') )
   {
      $sessiondata = array();
      $session_id = md5(d8ef2eab);
      $sessionmethod = SESSION_METHOD_GET;
   }

[Pablasso]

but i found that with posts with 2 or more pages simply doesnt work... when i tried to go to page #2 or other than #1, it always send me to page #1

this might happen if using (HTML compliant) & instead of & in your links...

Step 1. In /includes/page_header.php before

Code: Select all

//
// Generate logged in/logged out status
//

add this code (make sure there are no space breaks at line ends after you paste):

Code: Select all

//
// Short URL implementation
//
ob_start();
function replace_for_mod_rewrite(&$s)
{
$urlin =
array(
"'(?<!/)viewforum.php\?f=([0-9]*)&(?:amp;)topicdays=([0-9]*)&(?:amp;)start=([0-9]*)'",
"'(?<!/)viewforum.php\?f=([0-9]*)&(?:amp;)mark=topics'",
"'(?<!/)viewforum.php\?f=([0-9]*)'",
"'(?<!/)viewtopic.php\?t=([0-9]*)&(?:amp;)view=previous'",
"'(?<!/)viewtopic.php\?t=([0-9]*)&(?:amp;)view=next'",
"'(?<!/)viewtopic.php\?t=([0-9]*)&(?:amp;)postdays=([0-9]*)&(?:amp;)postorder=([a-zA-Z]*)&(?:amp;)start=([0-9]*)'",
"'(?<!/)viewtopic.php\?t=([0-9]*)&(?:amp;)start=([0-9]*)&(?:amp;)postdays=([0-9]*)&(?:amp;)postorder=([a-zA-Z]*)&(?:amp;)highlight=([a-zA-Z0-9]*)'",
"'(?<!/)viewtopic.php\?t=([0-9]*)&(?:amp;)start=([0-9]*)'",
"'(?<!/)viewtopic.php\?t=([0-9]*)'",
"'(?<!/)viewtopic.php&(?:amp;)p=([0-9]*)'",
"'(?<!/)viewtopic.php\?p=([0-9]*)'",
);
$urlout = array(
"viewforum\\1-\\2-\\3.html",
"forum\\1.html",
"forum\\1.html",
"ptopic\\1.html",
"ntopic\\1.html",
"ftopic\\1-\\2-\\3-\\4.html",
"ftopic\\1.html",
"ftopic\\1-\\2.html",
"ftopic\\1.html",
"sutra\\1.html",
"sutra\\1.html",
);
$s = preg_replace($urlin, $urlout, $s);
return $s;
}

it worked perfect!

thank you man!

[percussionforum]

has anybody tested this mod or made a mod for phpbb plus 1.3?

[mushy]

User-agent: * means every bot/browser/machine/thing that reads and acknowledges robots.txt

I tried this and as a result I am getting lesser traffic since I disallowed forum and topic to be indexed.....

Isn't there some good wroking and effective mod which disallow google, inkomi and other search engines to not to use multiple session. This thing is going into my nerves now I have only 200 sessions allowed by my host. Please somebody help me, pleaseeeeeeeeeeeeeeeee. I would be highly highly greatful......

[NeoThermic]

You should be looking towards this:
http://www.phpbb.com/kb/article.php?article_id=42
read the section entitled Limiting the number of sessions per IP

That should help.

NeoThermic

[mushy]

Well I don't know would it or not since it says at the bottom of that mod that

Code: Select all

Note that limiting sessions on a per IP basis is not suitable if you have many users behind a proxy. If you had twenty people behind one proxy gateway accessing the fora at the same time, then the total number of people acccessing the fora from behind that proxy would be limited, not the amount of connections one user creates on the fora.

There is code that limits the amount of sessions in a 60 second period already present in phpBB 2.2, the next major release of phpBB, but no anti-session flooding code will be added to phpBB 2.0.x.

And one more thing is I am using 2.0.6 and it says controlling such session will be part of major release after 2.0.2..

[mushy]

I just want things for spiders and not the users.... The mod about which the thread is about, people say it's not good and will give lots of trouble. Isn't there something similar but effective and not giving problems.....

[gurukast]

this code here...

Code: Select all

$sql = "SELECT COUNT(*) as numrows FROM ". SESSIONS_TABLE ." WHERE session_ip = '$user_ip'";
        $result = $db->sql_query($sql);
        if ( !$result )
        {
            message_die(CRITICAL_ERROR, 'Error checking existing sessions', '', __LINE__, __FILE__, $sql);
        }
       
        $numrows = $db->sql_fetchrow($result);
        if($numrows['numrows'] > 4)
        {
            $delete_limit = $numrows['numrows'] - 4;
            switch( SQL_LAYER )
            {
                case 'mysql4':
                    $sql = "DELETE FROM ". SESSIONS_TABLE ." WHERE session_ip = '$user_ip' ORDER BY session_start ASC LIMIT $delete_limit";
                    break;
                default:
                    $sql = "SELECT session_start FROM ". SESSIONS_TABLE ." WHERE session_ip = '$user_ip' ORDER BY session_start DESC LIMIT 4";
                    $result = $db->sql_query($sql);
                   
                    if ( !$db->sql_query($sql) )
                    {
                        message_die(CRITICAL_ERROR, 'Error select session data', '', __LINE__, __FILE__, $sql);
                    }
                   
                    $session_rows = $db->sql_fetchrowset($result);
                   
                    $sql = "DELETE FROM ". SESSIONS_TABLE ." WHERE session_ip = '$user_ip' AND session_start < ".$session_rows[3]['session_start'];
            }
                   
            if ( !$db->sql_query($sql) )
            {
                message_die(CRITICAL_ERROR, 'Error deleting old sessions', '', __LINE__, __FILE__, $sql);
            }
        }

hmmm so if we adapt this to the agent mod....