robots.txt

Do not post support requests, bug reports or feature requests. Discuss phpBB here. Non-phpBB related discussion goes in General Discussion!
Get Involved
stocksduniya
Registered User
Posts: 11
Joined: Thu Oct 15, 2009 12:39 pm
Contact:

robots.txt

Post by stocksduniya » Sat Nov 07, 2009 9:18 am

Dear Friends,
:geek: What about robots.txt ?

I wanna create robots.txt but I have not idea for phpBB robots.txt
Can anybody help me pease?

Thank you!

User avatar
ameeck
Former Team Member
Posts: 6559
Joined: Mon Mar 21, 2005 6:57 pm

Re: robots.txt

Post by ameeck » Sat Nov 07, 2009 9:28 am

Actually you shouldn't need it. Bots do not see links to non-content pages.

stocksduniya
Registered User
Posts: 11
Joined: Thu Oct 15, 2009 12:39 pm
Contact:

Re: robots.txt

Post by stocksduniya » Sat Nov 07, 2009 11:54 am

Thank you for guide me.

craiga
Registered User
Posts: 191
Joined: Fri Apr 10, 2009 8:14 am
Contact:

Re: robots.txt

Post by craiga » Sat Nov 07, 2009 3:08 pm

That is an interesting point about not needing a robots.txt due to the way bots are handled in phpbb.

I use a robots.txt, but the main reason is so that i can include the path to my sitemap.php file

stocksduniya
Registered User
Posts: 11
Joined: Thu Oct 15, 2009 12:39 pm
Contact:

Re: robots.txt

Post by stocksduniya » Sun Nov 08, 2009 12:40 pm

We are submitting our sitemap via 'Google Web Master Tools' but another search engines not providing this type of services. We should to use robots.txt or not? :cry:

User avatar
Sam
Former Team Member
Posts: 2082
Joined: Thu Jan 22, 2009 3:59 am
Location: Sacramento, CA
Name: Sam Thompson
Contact:

Re: robots.txt

Post by Sam » Mon Nov 09, 2009 12:00 am

stocksduniya wrote:We are submitting our sitemap via 'Google Web Master Tools' but another search engines not providing this type of services. We should to use robots.txt or not? :cry:
If you want bots to pickup your sitemap, then simply make a robots.txt. Robots.txt is often used to tell robots where to look (and more often where not to look) for content. Expanding on what ameeck said, you wont need one if you are restring pages to bots, such as telling them not to view files that do not have content; phpBB handles that itself by not linking to non content pages in the first place. If you want to link your sitemap, then you will need a robots.txt.

stocksduniya
Registered User
Posts: 11
Joined: Thu Oct 15, 2009 12:39 pm
Contact:

Re: robots.txt

Post by stocksduniya » Mon Nov 09, 2009 10:21 pm

:D Thank you for guide me!

User avatar
Lumpy Burgertushie
Registered User
Posts: 66332
Joined: Mon May 02, 2005 3:11 am
Contact:

Re: robots.txt

Post by Lumpy Burgertushie » Wed Nov 11, 2009 12:56 am

and you don't really need a sitemap for dynamic sites like phpbb anyway.

the links change on a daily basis so it is not really giving the search engines an up to date list of pages etc.


robert
I'm baaaaaccckkkk. still doing work on donation basis. PM your needs.

Premium phpBB 3.2 Styles by PlanetStyles.net

If a tree falls in the forest and nobody is there, does it make a sound?

stocksduniya
Registered User
Posts: 11
Joined: Thu Oct 15, 2009 12:39 pm
Contact:

Re: robots.txt

Post by stocksduniya » Wed Nov 11, 2009 10:46 am

Hmmm...

Thank you!
Lumpy Burgertushie wrote:and you don't really need a sitemap for dynamic sites like phpbb anyway.

the links change on a daily basis so it is not really giving the search engines an up to date list of pages etc.


robert

Pony99CA
Registered User
Posts: 4783
Joined: Thu Sep 30, 2004 3:13 pm
Location: Hollister, CA
Name: Steve
Contact:

Re: robots.txt

Post by Pony99CA » Wed Nov 11, 2009 10:59 pm

Lumpy Burgertushie wrote:and you don't really need a sitemap for dynamic sites like phpbb anyway.

the links change on a daily basis so it is not really giving the search engines an up to date list of pages etc.
Wouldn't giving a sitemap to the forums be useful? Those aren't likely to change too often.

Steve
Silicon Valley Pocket PC (http://www.svpocketpc.com)
Creator of manage_bots and spoof_user (ask me)
Need hosting for a small forum with full cPanel & MySQL access? Contact me or PM me.

User avatar
Sam
Former Team Member
Posts: 2082
Joined: Thu Jan 22, 2009 3:59 am
Location: Sacramento, CA
Name: Sam Thompson
Contact:

Re: robots.txt

Post by Sam » Thu Nov 12, 2009 8:36 am

Pony99CA wrote:
Lumpy Burgertushie wrote:and you don't really need a sitemap for dynamic sites like phpbb anyway.

the links change on a daily basis so it is not really giving the search engines an up to date list of pages etc.
Wouldn't giving a sitemap to the forums be useful? Those aren't likely to change too often.

Steve
Google does fine without it.

User avatar
Lumpy Burgertushie
Registered User
Posts: 66332
Joined: Mon May 02, 2005 3:11 am
Contact:

Re: robots.txt

Post by Lumpy Burgertushie » Thu Nov 12, 2009 2:14 pm

and I keep coming back to this question;
if your site/board is not trying to sell something then why is placement in google so important?

robert
I'm baaaaaccckkkk. still doing work on donation basis. PM your needs.

Premium phpBB 3.2 Styles by PlanetStyles.net

If a tree falls in the forest and nobody is there, does it make a sound?

User avatar
Eelke
QA Team
Posts: 2903
Joined: Thu Dec 20, 2001 8:00 am
Location: NL, Bussum
Name: Eelke Blok
Contact:

Re: robots.txt

Post by Eelke » Thu Nov 12, 2009 2:45 pm

Well, first of all "sell" is a relative term. Even if you only have a few advertisements, you're "selling" something. Namely, advertisement space. Plus, there are more incentives than making money. If you're in a competitive market, even if it is purely "for fun", it's nice to beat the competition in terms of site visitors.

User avatar
Lumpy Burgertushie
Registered User
Posts: 66332
Joined: Mon May 02, 2005 3:11 am
Contact:

Re: robots.txt

Post by Lumpy Burgertushie » Thu Nov 12, 2009 11:29 pm

Eelke wrote:Well, first of all "sell" is a relative term. Even if you only have a few advertisements, you're "selling" something. Namely, advertisement space. Plus, there are more incentives than making money. If you're in a competitive market, even if it is purely "for fun", it's nice to beat the competition in terms of site visitors.
well, that is what I meant. if you are running ads then it makes sense to try to get the most visitors that you can.

however, I don't think that most people that ask about getting indexed in google are really running any ads or otherwise trying to sell something.

from what I have seen, people seem to think that just because you have a website/board, that you should worry about getting high page rank and/or position in google.

as far as it being "fun" to get more visitors than your "competition" how would you or they know?
and what is "competition" when you are talking about a game related board, or a board about tie collecting or whatever?

I am not trying to be argumentative here, I just wonder why people worry so much about this type of thing when it really doesn't matter in the long run.

robert
I'm baaaaaccckkkk. still doing work on donation basis. PM your needs.

Premium phpBB 3.2 Styles by PlanetStyles.net

If a tree falls in the forest and nobody is there, does it make a sound?

User avatar
Sam
Former Team Member
Posts: 2082
Joined: Thu Jan 22, 2009 3:59 am
Location: Sacramento, CA
Name: Sam Thompson
Contact:

Re: robots.txt

Post by Sam » Fri Nov 13, 2009 2:25 am

I could honestly care less about page rank, I like running the boards because I like my userbase, and they call the boards I run home. The only thing I really care about is that ad revenue is at the very least equal to the hosting/domain costs.
Last edited by Sam on Mon Nov 16, 2009 5:38 am, edited 1 time in total.

Post Reply

Return to “phpBB Discussion”