Will Google consider it cloaking if I give bot access to a forum that doesn't have guest access?

Do not post support requests, bug reports or feature requests. Discuss phpBB here. Non-phpBB related discussion goes in General Discussion!
Scam Warning
Post Reply
phplessthannovice
Registered User
Posts: 6
Joined: Sun May 24, 2020 9:32 am

Will Google consider it cloaking if I give bot access to a forum that doesn't have guest access?

Post by phplessthannovice »

I couldn't find any clear information on this. What types of gated content are considered okay (if any)? Do I have to go through all the forums and make sure that those that don't have guest access also don't have bot access?
User avatar
thecoalman
Community Team Member
Community Team Member
Posts: 3910
Joined: Wed Dec 22, 2004 3:52 am
Location: Pennsylvania, U.S.A.
Contact:

Re: Will Google consider it cloaking if I give bot access to a forum that doesn't have guest access?

Post by thecoalman »

If you are serving content to Google that is not available for guests it's going to be considered cloaking with one exception. You can indicate to Google it's behind a login/subscription/paywall.

https://developers.google.com/search/do ... ed-content

One other thing to keep in mind, phpBB utilizes the user agent to identify bots and it can be spoofed.
“Results! Why, man, I have gotten a lot of results! I have found several thousand things that won’t work.”

Attributed - Thomas Edison
User avatar
mrgtb
Registered User
Posts: 603
Joined: Wed Oct 03, 2007 10:51 am

Re: Will Google consider it cloaking if I give bot access to a forum that doesn't have guest access?

Post by mrgtb »

thecoalman wrote:
Wed Jul 22, 2020 12:06 am
One other thing to keep in mind, phpBB utilizes the user agent to identify bots and it can be spoofed.
That's a good point. if you're going to block guests from seeing something, then you really should block bots also with them. As said, it's not hard for a user to spoof themselves as a google bot and still read what guests cannot read. There are even browser plug-ins to do it I hear, making it a simple thing to do.
User avatar
thecoalman
Community Team Member
Community Team Member
Posts: 3910
Joined: Wed Dec 22, 2004 3:52 am
Location: Pennsylvania, U.S.A.
Contact:

Re: Will Google consider it cloaking if I give bot access to a forum that doesn't have guest access?

Post by thecoalman »

mrgtb wrote:
Wed Jul 22, 2020 4:18 pm
That's a good point. if you're going to block guests from seeing something, then you really should block bots also with them.
That depends, if you aren't really trying to protect the content it doesn't matter. I had one forum for example that was behind login and I didn't want indexed either, I was allowing in just Googles Adbots because I was serving ads in it. I had special group called ad bots for the ad bot. I wasn't really concerned if someone spoofed it because they only needed to register to see it.

phpBB can use an IP to identify bots which will avoid spoofing issue. If you were going to allow them to index content that must not be seen by guests you can set up special group called Bots by IP and add those bots to it where you have added the IP. You can allow access for ones identified by IP and deny for other bots in the regular bots groups where you have not added an IP indetifier.
“Results! Why, man, I have gotten a lot of results! I have found several thousand things that won’t work.”

Attributed - Thomas Edison
User avatar
mrgtb
Registered User
Posts: 603
Joined: Wed Oct 03, 2007 10:51 am

Re: Will Google consider it cloaking if I give bot access to a forum that doesn't have guest access?

Post by mrgtb »

Yeah, if you use Google AdSense, then you don't have much option but to still allow that bot in to serve the ads.

Adding IP's for bots. Yeah, I know you can do that but not a great idea doing it instead of adding the user agent. Pretty sure even adsense bots will hit you site using multiple different IPs.
Post Reply

Return to “phpBB Discussion”