Googlebot is logged in as registered user!

k7a · Post by **k7a** » Wed Oct 07, 2009 9:55 pm

I run a PHPBB3 board, where guests are not allowed to see anything (only registered users are allowed to read/post, and only admins can register/activate new accounts).

Now, suddenly many threads were indexed at Google. Wow, what happened? Guests are not allowed to access the content, and all bots are "deactivated". And this is, as I mentioned, a private forum. And now it's all in Google.

If I open the indexed threads in Google Cache, you can clearly see, that the bot was logged in.
It says "Your last visit was ..." at the top, and it offers to show "New Posts" and "Own posts". And at "Who's online" it says: John Doe and 0 guests. Holy shit? John Doe is a registered user at this forum.

And this John Doe was online very often online in the last days/weeks, even at night - and even when the "real" John Doe was sleeping and his PC off. Now, I clicked at this account at "Who's online" (while the real John Doe was not logged in at his PC), and it says:

IP: 66.249.65.42 » Whois
Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)

And if I click at "Whois":

OrgName: Google Inc.
OrgID: GOGL
Address: 1600 Amphitheatre Parkway
City: Mountain View
StateProv: CA
PostalCode: 94043
Country: US

NetRange: 66.249.64.0 - 66.249.95.255
CIDR: 66.249.64.0/19
NetName: GOOGLE
NetHandle: NET-66-249-64-0-1
Parent: NET-66-0-0-0-0
NetType: Direct Allocation
NameServer: NS1.GOOGLE.COM
NameServer: NS2.GOOGLE.COM
NameServer: NS3.GOOGLE.COM
NameServer: NS4.GOOGLE.COM
Comment:
RegDate: 2004-03-05
Updated: 2007-04-10

OrgTechHandle: ZG39-ARIN
OrgTechName: Google Inc.
OrgTechPhone: +1-650-318-0200
OrgTechEmail: arin-contact@google.com

# ARIN WHOIS database, last updated 2009-10-06 20:00
# Enter ? for additional hints on searching ARIN's WHOIS database.

Oh great, the Googlebot is using an users board account to index threads? Hello? How could this be possible?

Now, I (admin) deactivated the account and changed the password, and the real John Doe logged in again (this time with Firefox, before with Google Chrome). It worked. He logged out again. And some minutes later, the "fake" John Doe (= googlebot?) came again. Wow, why could he still login? (session or something?) However, I cleared the forums cache, changed the password again - and then the bot was logged out (and he became a guest). I banned 66.249.65.42 at admin panel.

Now, maybe John Does PC is infected. But how could it be, that the googlebot (is this the real googlebot, btw?) uses his account to index our board?! I mean, if it would be some kind of spyware/trojan, okay, but how does it come to Googlebot?
Might this be a "function" (wow, what a feature ...?) of Google Chrome?

k7a · Post by **k7a** » Thu Oct 08, 2009 11:06 am

What should I do now? Anything to log, before it might get deleted?

Is this a problem with PHPBB (assigning wrong account to googlebot?) or with Google? (stealing/borrowing username/password?)

Pit$Bull · Post by **Pit$Bull** » Thu Oct 08, 2009 11:15 am

Instead of "deactivated" you should set the bot group to no access.
ACP->permissions tab->Groups’ forum permissions->choose bots and set permissions to no access for the forums you want private.

k7a · Post by **k7a** » Thu Oct 08, 2009 12:26 pm

Pit$Bull wrote:Instead of "deactivated" you should set the bot group to no access.

In forum permissions, the bot group had "no role assigned" (I changed it now to "no access", but 'no role assigned' means the same, doesn't it?).
However, I mean ACP -> System -> Spiders/Robots. All these bots are "deactivated", and their last visit is "never".

So, this was no security breach, correct? And it doesn't explain the situation, that the googlebot was logged in as another registered user ... that the bot grabbed this account.

Pit$Bull · Post by **Pit$Bull** » Thu Oct 08, 2009 1:08 pm

The bots should be activated in order to have complete control. "no role assigned" will leave you with the option to specify your desires with advanced permissions. If you want no access for bots, guests, etc, then use no access.

Pit$Bull wrote: ACP->permissions tab->Groups forum permissions->choose group and set permissions to no access for the forums you want private.

I have no idea what you have done to cause the other issue but will suggest checking ALL permissions using the permission mask option.
Knowledge Base - phpBB3 Permissions

Post by **Brf** » Thu Oct 08, 2009 1:17 pm

k7a wrote:Oh great, the Googlebot is using an users board account to index threads? Hello? How could this be possible?

This kind of thing happens in a bad conversion, where the Bots are given user_id's in the user range.
If you use phpmyAdmin, or some other database tool, and browse the phpbb_users table you should see the Bots user_ids in their proper range. Browse the phpbb_bots table and set their user_id's to the proper values.

k7a · Post by **k7a** » Thu Oct 08, 2009 2:50 pm

Pit$Bull wrote:The bots should be activated in order to have complete control.

I assigned all bots the role "no access". So I should activate all bots now, too?

I have no idea what you have done to cause the other issue but will suggest checking ALL permissions using the permission mask option.

I checked all permissions. Everything was okay. I mean ... Google did only have access, because it used an user account! So there seems to be no access/rights problem, but the problem, why Google "borrows"/steals an user account.

Brf wrote: This kind of thing happens in a bad conversion, where the Bots are given user_id's in the user range.
If you use phpmyAdmin, or some other database tool, and browse the phpbb_users table you should see the Bots user_ids in their proper range. Browse the phpbb_bots table and set their user_id's to the proper values.

Hm, could you provide further details/explanations, please?
So the googlebot got the user id of a real registered user? Why/How? And so everyone, who changes his user-agent to the one of googlebot, would have access to the forum?

Hm, I'm now at phpMyAdmin and looking at the phpbb_users and the phpbb_bots table. What exactly should I do now? The googlebot has user id 66 at phpbb_bots and at phpbb_users, too. The user account of John Doe got ID 16.

Should/could I just delete all bot accounts, would this solve the problem? (I mean .. I don't need them, do I? As I said, it's a private forum)

Post by **Brf** » Fri Oct 09, 2009 12:16 am

Googlebot is normally user_id 16. I am wondering why it would be accessing user-16, it it has a different user_id in the bots table.

Have yo cleared your cache lately?

k7a · Post by **k7a** » Fri Oct 09, 2009 2:45 am

Brf wrote:Have yo cleared your cache lately?

I cleared it after changing the password for John Doe (because the googlebot was still logged in as John Doe, although the password was changed)

You mean, I should clear it again and check, if the user ids change? (err... could they?)
Clearing the forum cache at ACP, correct?

Post by **Brf** » Fri Oct 09, 2009 11:33 am

If GoogleBot's user_id in the phpbb_bots table is not 16, it should not be using user 16. I thought perhaps it was using a cached version of the bots table that still had 16 in that user_id.

k7a · Post by **k7a** » Thu Oct 15, 2009 5:36 pm

Well, hm, what should I do now? How can I prevent this? I want a closed forum, not a publicly available forum, for those who use the "correct" user-agent (I mean ... is this not critical?!)

Could I delete all bots via ACP?

Post by **Brf** » Thu Oct 15, 2009 5:43 pm

Dont delete them. Disable them.

k7a · Post by **k7a** » Thu Oct 15, 2009 5:48 pm

Brf wrote:Disable them.

Well, they were deactivated, when the problem occurred. But Google became a regular user, so none of the bot rules fit, I think.

Again, if it isn't clear:
there is only one group, which has read/post access to the forum. And in this group are only hand chosen registered users. And now the Googlebot took over one of those user accounts. He logged in, visited some threads and indexed them at Googles Search Engine, logged in as this registered user!

Post by **Brf** » Thu Oct 15, 2009 5:53 pm

If a bot is deactivated, it cannot do anything. Are you sure you deactivated it?

k7a · Post by **k7a** » Thu Oct 15, 2009 7:57 pm

Brf wrote:If a bot is deactivated, it cannot do anything. Are you sure you deactivated it?

Yes, very sure!
And beside that: there is only ONE group which has read-access, and this group has 15 members only and none of them is a bot

As I described, the Googlebot was not logged in as Googlebot, but as John Doe (where John Doe is a regular, human user!!).

"John Doe" (the Googlebot) was logged in very often at night. He had the Googlebot ip, the Googlebot user-agent (I posted it in this thread) AND it indexed the threads. I described it, too: if you open the indexed threads in Google Cache, you can see that there is always John Doe logged in (and no other members).

So the Googlebot browsed to our forum, and was logged in as "John Doe" immediately. And so it could see anything, that John Doe is allowed to -- and John Doe is allowed to see the forum. == very bad, because now all our "private" threads are published in Google Cache.

advertisements