Adding noindex, follow to some pages.

Get help with installation and running phpBB 3.0.x here. Please do not post bug reports, feature requests, or MOD-related questions here.
Scam Warning
Forum rules
END OF SUPPORT: 1 January 2017 (announcement)
Locked
Thanks in advance
Registered User
Posts: 173
Joined: Mon Dec 28, 2009 8:17 am

Adding noindex, follow to some pages.

Post by Thanks in advance »

I think instead of just blocking access for search engine spiders to some pages like search.php, faq.php... it would be better to add the "noindex, follow" tag. I guess this has to be done in the includes / function.php, but I don't know how. Anyone wiser out there?
User avatar
Rahber
Former Team Member
Posts: 2720
Joined: Tue Feb 12, 2008 3:39 pm
Location: Pakistan
Name: Rahber
Contact:

Re: Adding noindex, follow to some pages.

Post by Rahber »

when you are talking about the non index tag it has to be meta tag

and that tag is

Code: Select all

<meta name="robots" content="noindex" />
the other way is adding the files name to disallow in robots.txt as all major robots follow robots.txt
Thanks in advance
Registered User
Posts: 173
Joined: Mon Dec 28, 2009 8:17 am

Re: Adding noindex, follow to some pages.

Post by Thanks in advance »

Thanks for the answer. I know how to use the tag in html-files and about robots.txt, but how can I manage that "noindex, follow" is used in some less important php-files?
User avatar
stevemaury
Support Team Member
Support Team Member
Posts: 51681
Joined: Thu Nov 02, 2006 12:21 am
Location: The U.P.
Name: Steve
Contact:

Re: Adding noindex, follow to some pages.

Post by stevemaury »

BOTS don't know about php files, they crawl content displayed by html files. If you do not want bots to see, for example, the FAQ, use this switch in the template:

Code: Select all

<!-- IF not S_IS_BOT -->what you do not want them to see<!--ENDIF -->
For REALLY good and VERY inexpensive hosting CLICK HERE

I can stop all your spam. I can upgrade or update your Board. PM or email me. (Paid support)
Thanks in advance
Registered User
Posts: 173
Joined: Mon Dec 28, 2009 8:17 am

Re: Adding noindex, follow to some pages.

Post by Thanks in advance »

stevemaury wrote:BOTS don't know about php files, they crawl content displayed by html files. If you do not want bots to see, for example, the FAQ, use this switch in the template:

Code: Select all

<!-- IF not S_IS_BOT -->what you do not want them to see<!--ENDIF -->
Cloaking refers to the practice of presenting different content or URLs to users and search engines. Serving up different results based on user agent may cause your site to be perceived as deceptive and removed from the Google index.
http://www.google.com/support/webmaster ... swer=66355

Does anybody experienced know what code to add to e.g. the faq-file so that the resulting / generated html-page has the noindex, follow meta tag in it?
jack616
Registered User
Posts: 29
Joined: Wed Jun 15, 2011 4:35 pm

Re: Adding noindex, follow to some pages.

Post by jack616 »

I just block google completely now.
I have three web sites and on all 3 they randomly over a period of months ignore the robots.txt file.
I know its deliberate because it is random and no changes are made my end.
All other robots function fine. Even yahoo seems honest.

I'd suggest you do what I do now - let google index a page or two including your
front page then kill them completely in the .htaccess file using quad blocks.

I've heard all the excuses under the sun for this. Just dont expect robots.txt
to be obeyed by google is all I say. Keep an eye on your logs. They do this randomly.

And dont forget they use multiple robots for dfifferent purposes.
Do make sure its google causing the trouble though - hackers can make themselves
appear to be google. You need to zap them too of course.

In addition to noindex you may also want to try noarchive and nofollow.
If google is ignoring those too I wouldn't be at all suprised.

jack616
IanSn
Registered User
Posts: 62
Joined: Tue Apr 14, 2009 2:44 am
Location: London, UK

Re: Adding noindex, follow to some pages.

Post by IanSn »

jack616 wrote: Just dont expect robots.txt
to be obeyed by google is all I say.
Microsoft are the main offenders for this. Bing takes no notice of the robots file at all. But MS never has. They also ignore 'nofollow', etc.
But I agree - I block 'em.
Certain google IPs are well behaved, others are not. I selectively block. (On my personal firewall as well.)

I know people love google, but I don't. Seriously fed up of them in every damn thing everywhere - in and out of your computer with everything you do everywhere you go.

Reminds me of the film 'Brazil' where those machines follow you around in your face and poking your pockets everywhere you walk. Infuriating.... :evil:

Oops, drifting off topic....
Thanks in advance
Registered User
Posts: 173
Joined: Mon Dec 28, 2009 8:17 am

Re: Adding noindex, follow to some pages.

Post by Thanks in advance »

Why on earth would I block Google from indexing my site? To get much less visitors and money through ads?
IanSn
Registered User
Posts: 62
Joined: Tue Apr 14, 2009 2:44 am
Location: London, UK

Re: Adding noindex, follow to some pages.

Post by IanSn »

I use non-board html 'portals' with relevant meta-data, etc., as the main hit and link from there to the board on a different domain that blocks all bots.
Among other things...
Thanks in advance wrote:Why on earth would I block Google from indexing my site? To get much less visitors and money through ads?
Hey, depends on what sort of site it is.

For us, lots of visitors are just a pain, filling the place with pointless, unhelpful comments, people trying to sell the researchers something they don't want or need, and requiring lots of moderation.

Weird?
No, not if you are a forum for, say, a specific academic subject, trying to keep the data/info making sense, relevent, quick, uncluttered and useful to researchers. Allowing proper discussions to take place without threads drifting off topic..... ahem :roll:

Using the search function on a clean and reliable site is preferable (in the opinion of some) to pulling up a thousand pages on google, some of which exist soley to get your money, or have some other agenda.

So, yep, some folks are interested in 'no-index' and rightly annoyed when google, etc., hit the disallowed files/directories anyway.

Ads? Don't use 'em on forums. Just annoying.
I particularly dislike some marketing oik targeting (pigeon-holing) me and "all my friends".

ian
Thanks in advance
Registered User
Posts: 173
Joined: Mon Dec 28, 2009 8:17 am

Re: Adding noindex, follow to some pages.

Post by Thanks in advance »

I found a way to add those meta tags to phpBB-files. Could someone here (mayb with some SEO-experience) please tell me for which files "noindex, follow" is perfect and for which "noindex, nofollow"?
User avatar
Lumpy Burgertushie
Registered User
Posts: 68296
Joined: Mon May 02, 2005 3:11 am
Contact:

Re: Adding noindex, follow to some pages.

Post by Lumpy Burgertushie »

Thanks in advance wrote:I found a way to add those meta tags to phpBB-files. Could someone here (mayb with some SEO-experience) please tell me for which files "noindex, follow" is perfect and for which "noindex, nofollow"?
why are you worrying about this? the standard setup seems to work just fine for thousands of boards around the world.

if you put a meta tag in the overall_header file it will affect every page of the board.

other than that I don't know what you are wanting to do.

robert
I'm baaaaaccckkkk. still doing work on donation basis. PM your needs.

Premium phpBB 3.3 Styles by PlanetStyles.net

If nobody is in the forest, does a tree really fall?
professor mauricio
Registered User
Posts: 30
Joined: Tue Jul 12, 2011 4:54 pm

Re: Adding noindex, follow to some pages.

Post by professor mauricio »

Sorry, but i have to repost because i've posted in a wrong place:
***
My tip is

Code: Select all

<!-- IF TOPIC_ID <1 --><meta name="robots" content="noindex" /><!-- ENDIF -->
Putting this code close to meta tags of the header (before </head>) bots indexes just pages that has topics

it means just ./viewtopic.php parsed to existing topics

i'm using succesfuly something like this

Should I call this the mini-micro-MOD of SEO?

um abraço
prof
Bushwhacked
Registered User
Posts: 15
Joined: Wed Feb 10, 2010 9:46 am

Re: Adding noindex, follow to some pages.

Post by Bushwhacked »

Lumpy Burgertushie wrote: why are you worrying about this? the standard setup seems to work just fine for thousands of boards around the world.
Answer: because I want to do something different that what the "standard setup" allows.

The question was/is: how can one add a noindex/nofollow tag one particular post? NOT the whole board, JUST ONE post. For example, is there any way to noindex the following post:

http://{www.siteurl.com}/viewtopic.php?f=12&t=297

Is there an if/then meta tag that can be inserted in viewtopicbody, or in the head, that will "switch off" indexing for JUST this one post? I am obviously aware that the post can just be deleted, but I'd prefer not to do that.

Thanks in advance for any constructive thoughts/ideas.
User avatar
Oyabun1
Former Team Member
Posts: 23162
Joined: Sun May 17, 2009 1:05 pm
Location: Australia
Name: Bill

Re: Adding noindex, follow to some pages.

Post by Oyabun1 »

Bushwhacked wrote:Is there an if/then meta tag that can be inserted in viewtopicbody, or in the head, that will "switch off" indexing for JUST this one post? I am obviously aware that the post can just be deleted, but I'd prefer not to do that.
There are no existing IFs or META tags in phpBB that could do this.

There are a few options:
  • Create a custom BBCode to be used for "no follow" links, something like:
    BBCode

    Code: Select all

    [nofollow={URL}]{TEXT}[/link][/nofollow]
    HTML

    Code: Select all

    <a href="{URL}" rel="nofollow">{TEXT}</a>
  • Use a MOD. Prime Links, allows certain domains to be set as no follow.
  • Request a MOD in [3.0.x] MOD Requests.
                      Support Request Template
3.0.x: Knowledge Base Styles Support MOD Requests
3.1.x: Knowledge BaseStyles SupportExtension Requests
Locked

Return to “[3.0.x] Support Forum”