Disable the ?sid= from url for google seo spider

Do not post support requests, bug reports or feature requests. Discuss phpBB here. Non-phpBB related discussion goes in General Discussion!
Anti-Spam Guide
Post Reply
eselva
Registered User
Posts: 8
Joined: Tue Oct 16, 2018 11:43 am

Disable the ?sid= from url for google seo spider

Post by eselva » Sat Jun 01, 2019 2:56 pm

Hello, im trying to optimize my forum for seo purposes but I've noticed that the bots take the same url repeated hundreds of times because of the variable sid, how can i disable it? I tried to disallow the sid on robots.txt, but then it disallows the whole /viewtopic.php urls . I think my cookies config on the adm are correct.

User avatar
stevemaury
Support Team Member
Support Team Member
Posts: 50506
Joined: Thu Nov 02, 2006 12:21 am
Location: The U.P.
Name: Steve
Contact:

Re: Disable the ?sid= from url for google seo spider

Post by stevemaury » Sat Jun 01, 2019 11:30 pm

Post a link to your board and post your cookie settings. Also, is GoogleBOT in the Bots group?
For REALLY good and VERY inexpensive hosting CLICK HERE

I can stop all your spam. I can upgrade or update your Board. PM or email me. (Paid support)

eselva
Registered User
Posts: 8
Joined: Tue Oct 16, 2018 11:43 am

Re: Disable the ?sid= from url for google seo spider

Post by eselva » Sun Jun 02, 2019 11:40 pm

stevemaury wrote:
Sat Jun 01, 2019 11:30 pm
Post a link to your board and post your cookie settings. Also, is GoogleBOT in the Bots group?
Thanks for the reply!

Yes, it is. But im using semrush to emulate google crawler and i just noticed that the semrush bot is not there. I will add it and then check if that was the problem...


https://blaugranas.es/

Image

User avatar
Brf
Support Team Member
Support Team Member
Posts: 51726
Joined: Tue May 10, 2005 7:47 pm
Location: {postrow.POSTER_FROM}
Contact:

Re: Disable the ?sid= from url for google seo spider

Post by Brf » Sun Jun 02, 2019 11:54 pm

When I look up your site on Google, I am not seeing any sids in any of the URLs.

User avatar
thecoalman
Community Team Member
Community Team Member
Posts: 3223
Joined: Wed Dec 22, 2004 3:52 am
Location: Pennsylvania, U.S.A.
Contact:

Re: Disable the ?sid= from url for google seo spider

Post by thecoalman » Mon Jun 03, 2019 1:57 am

Brf wrote:
Sun Jun 02, 2019 11:54 pm
When I look up your site on Google, I am not seeing any sids in any of the URLs.
They would be tagged as duplicate content and not appear in the search results if they were already indexed.

eselva
Registered User
Posts: 8
Joined: Tue Oct 16, 2018 11:43 am

Re: Disable the ?sid= from url for google seo spider

Post by eselva » Mon Jun 03, 2019 9:01 am

thecoalman wrote:
Mon Jun 03, 2019 1:57 am
Brf wrote:
Sun Jun 02, 2019 11:54 pm
When I look up your site on Google, I am not seeing any sids in any of the URLs.
They would be tagged as duplicate content and not appear in the search results if they were already indexed.
stevemaury wrote:
Sat Jun 01, 2019 11:30 pm
Post a link to your board and post your cookie settings. Also, is GoogleBOT in the Bots group?
Brf wrote:
Sun Jun 02, 2019 11:54 pm
When I look up your site on Google, I am not seeing any sids in any of the URLs.
Thanks for replies, I solved this issue now. From the beginning wasn't a problem with google, but with the bot I used to simulate the google bot( semrush). If you don't add it to the bot list, logically it takes sid in the url, in this case semrush uses 2 bots and multiplied the whole site x2 . Sorry, that was easy to solve :roll: .

There is the actual configuration if you want to use semrush to optimize your phpbb seo (im not sure if im able to post here the ip credentials, remove it if not):

SEMrush 66
SemrushBot
46.229.173.66

-
SEMrush 68
SemrushBot
46.229.173.68

Post Reply

Return to “phpBB Discussion”