Page 1 of 1

Anyone has experience with sitemaps xml?

Posted: Wed Jan 06, 2010 2:05 pm
I need help with sitemaps for the forum. It's very confusing of how to properly create/submit sitemap xml to Google.

I want to have sitemap that points to every individual post on the forum (no to thread, but to a single post)
A single sitemap will definetely be over the allowed 50,000 urls limit.
So I am thinking to create a new sitemap every day with posts that came it on that day. But that would mean that after a year I will have 365 sitemaps created.

Another strategy I was thinking about is to have monthly sitemap and keep modifying it every day and resubmitting it to Google (modified version daily), but that also means that modified sitemap will have only a small portion of its urls being new and the rest already know by Google.

This is so confusing. How do you do it? I've heard there is a hack for phpbb for this, but how do they do it? A new sitemap daily or just one sitemap?

Re: Anyone has experience with sitemaps xml?

Posted: Wed Jan 06, 2010 3:24 pm
by stevemaury
Exactly the reason why sitemaps are neither necessary nor effective with dynamic content. Phpbb gets indexed just fine without sitemaps.

Re: Anyone has experience with sitemaps xml?

Posted: Wed Jan 06, 2010 3:33 pm
I disagree that it gets indexed well. I mean, sure some posts get indexed well, but many posts are not getting in the index at all. I think Google now always following to page 2, 3, 4... in the threads page. This may depend on the PR of your forum, but if your PR is low, then Google will most likely not crawl your pagination links and will only index threads that it can see on the threads page at the time it visits your forum

Re: Anyone has experience with sitemaps xml?

Posted: Wed Jan 06, 2010 3:40 pm
by AdamR
Sitemaps were not designed for the purpose of indexing every single possible page on a site. Their purpose, however, is for providing quick access to content which is "hidden" or obscurely placed. phpBB, and forums in general, do not fall into that category because the board, categories, forums, topics, and posts are structured already. Many search engines specifically cater their spiders to be able to crawl topics more effectively. For example, many search engines know when the last post in a topic was made and display that on the index. It wouldn't know that information if it didn't browse to the last page of the topic. ;)

That being said, in order to accomplish this, you would need a MODification. There is already an existing Sitemap MOD, though I'm not certain if it goes to the extent of indexing individual posts. You'd want to check with the MOD author in that topic.

- Adam

Re: Anyone has experience with sitemaps xml?

Posted: Thu Jan 07, 2010 1:31 pm
by Dogs and things
I believe a sitemap that includes a lastmod date helps search engines to index a board faster and more accurately. If they follow the sitemap they will immediately find the latest new content without having to index all topics everytime they reindex a board.