Solve this SEO Issue Once and For All

Do not post support requests, bug reports or feature requests. Discuss phpBB here. Non-phpBB related discussion goes in General Discussion!
Scam Warning
User avatar
thecoalman
Community Team Member
Community Team Member
Posts: 2650
Joined: Wed Dec 22, 2004 3:52 am
Location: Pennsylvania, U.S.A.
Contact:

Re: Solve this SEO Issue Once and For All

Post by thecoalman » Thu Jul 12, 2018 12:31 pm

AmigoJack wrote:
Thu Jul 12, 2018 12:15 pm
Frank Rizzo wrote:
Thu Jul 12, 2018 11:08 am
why do phpbb offer it
Times and search engines change, and phpBB is not that good in recognizing all bots either. I guess the main intent was to see bots at all when crawling a board, instead of ignoring them.
It's useful in different ways, for example the online list is more accurate. One way I have used it is to create additional adsbot group. This allows me to grant access to Googles adbot to bot restricted forums.

User avatar
Mick
Support Team Member
Support Team Member
Posts: 19535
Joined: Fri Aug 29, 2008 9:49 am
Location: Cardiff

Re: Solve this SEO Issue Once and For All

Post by Mick » Thu Jul 12, 2018 2:16 pm

So, you’re allowing certain bots to crawl where guests and some registered users have no access?
"The more connected we get the more alone we become" - Kyle Broflovski

There are no ‘threads’ in phpBB, they are topics.

User avatar
thecoalman
Community Team Member
Community Team Member
Posts: 2650
Joined: Wed Dec 22, 2004 3:52 am
Location: Pennsylvania, U.S.A.
Contact:

Re: Solve this SEO Issue Once and For All

Post by thecoalman » Thu Jul 12, 2018 2:56 pm

Mick wrote:
Thu Jul 12, 2018 2:16 pm
So, you’re allowing certain bots to crawl where guests and some registered users have no access?
No, I have a few forums that are blocked to bots through robots.txt and phpBB permissions. All are available to public with the exception of one that is available to registered users. I think I know where you are going with this and I'm not concerned with someone spoofing the user agent to get access since they don't need to do that to begin with.

I'm displaying ads on those forums so Googles Mediapartners-Google needs access to it. Otherwise it will serve lousy ads or maybe none at all. in robots.txt at the top you put this:

Code: Select all

User-agent: Mediapartners-Google
Allow: /
This bot is also given access to those forums through phpBB permissions denied to all other bots. Just so it's clear you can display ads behind a login, as long as they are content pages is really the only thing that matters. You can even provide this bot with login credentials in your adsense account if you needed to. You don't need to do that with phpBB because you can identify it. ;)

Post Reply

Return to “phpBB Discussion”

Who is online

Users browsing this forum: thecoalman and 37 guests