[RC4] GYM Sitemaps & RSS (aka mx Google Sitemaps)

A place for MOD Authors to post and receive feedback on MODs still in development. No MODs within this forum should be used within a live environment! No new topics are allowed in this forum.
Forum rules
READ: phpBB.com Board-Wide Rules and Regulations

IMPORTANT: MOD Development Forum rules

On February 1, 2009 this forum will be set to read only as part of retiring of phpBB2.
dcz
Registered User
Posts: 787
Joined: Sun Feb 13, 2005 5:37 am
Contact:

Post by dcz »

Well, concerning the mod rewrite for sitemap links, your can either submit sitemap.php or sitemaps.xml where ever they stand.

And, usually, fullpath are set in .htaccess, but in your case it seems not, so the correct rules that should apply for you would be the second try.

Now I think I see where it comes from.
Actually http://www.pathway2curis.com/forum/sitemaps.xml works very well.

The problem comes from the mod rewrite implemented, since it's also using rules with forum in it.

So you just need to put those rules before the others, so that the first match will be for sitemaps, where there cannot be doubt with the forum rewriterules. This should be enough.

++

phpBB SEO || phpBB3 SEO Premod || SEO phpBB3
GYM Sitemaps & RSS for phpBB3: GYM Sitemaps & RSS
SHH
Registered User
Posts: 52
Joined: Wed Feb 22, 2006 1:06 am
Contact:

Post by SHH »

So you just need to put those rules before the others, so that the first match will be for sitemaps, where there cannot be doubt with the forum rewriterules. This should be enough.


Like this

Code: Select all

RewriteEngine On 
RewriteCond %{HTTP_HOST} !^.*pathway2curis\.com [NC]
RewriteRule (.*) http://www.pathway2curis.com/forum/$1 [R=301,L]
 
RewriteRule ^sitemaps.xml$ sitemap.php [L]
RewriteRule ^mx-sitemap.xml$ sitemap.php?mx [L]
RewriteRule ^forum-sitemap-([0-9]+).xml$ sitemap.php?fid=$1 [L]
RewriteRule ^kb-sitemap-([0-9]+).xml$ sitemap.php?kbid=$1 [L]
RewriteRule ^sitemap-forum.xml$ sitemap.php?forum [L]
RewriteRule ^sitemap-kbcat.xml$ sitemap.php?kbcat [L]

RewriteRule ^forums.* index.php [L,NC]
RewriteRule ^post-([0-9]*).html&highlight=([a-zA-Z0-9]*) viewtopic.php?p=$1&highlight=$2 [L,NC]
RewriteRule ^post-([0-9]*).* viewtopic.php?p=$1 [L,NC]
RewriteRule ^view-poll([0-9]*)-([0-9]*)-([a-zA-Z]*).* 

exc....
it still shows up as

Code: Select all

<sitemap>
−
	<loc>
http://pathway2curis.com/forum/forum-sitemap-13.xml
</loc>
<lastmod>2006-03-21T14:50:28+00:00</lastmod>
</sitemap>
−
	<sitemap>
−
	<loc>
http://pathway2curis.com/forum/forum-sitemap-38.xml
</loc>
I need it to rewrite
http://pathway2curis.com/forum/forum-sitemap-38.xml

to say

http://www.pathway2curis.com/forum/forum-38.html


also does this mod take into account any threads?

thanks
SHH
Registered User
Posts: 52
Joined: Wed Feb 22, 2006 1:06 am
Contact:

Post by SHH »

Hi again,

Ok I changed this part of your code in your index.php and it works with the mod rewrite...

line 114-115 was

Code: Select all

// Either mod rewrite or not for forum sitemaps
				$forum_sitemap_urls = ($Rewrite_sitemap) ? $root_url.'forum-sitemap-'.$forum_id.'.xml' : $root_url.'sitemap.'.$phpEx.'?fid='.$forum_id;
Changed to

Code: Select all

				// Either mod rewrite or not for forum sitemaps
				$forum_sitemap_urls = ($Rewrite_sitemap) ? $root_url.'forum-'.$forum_id.'.html' : $root_url.'sitemap.'.$phpEx.'?fid='.$forum_id;

Still not sure how to get the the threads to show up on the the sitemap

http://www.pathway2curis.com/forum/sitemap.php
SHH
Registered User
Posts: 52
Joined: Wed Feb 22, 2006 1:06 am
Contact:

Post by SHH »

A few minor errors in the ACP

In the ACP

"Sort Order
All outputed (outputted) links are sorted in the same way topics are sorted by default in phpbb (last activity DESC).
You can set this to DESC for example if you whish to make it easyer(easier) for Google to find again links to archeological or locked threads (eg inactive for a looong time)."

Cheers
SHH
Registered User
Posts: 52
Joined: Wed Feb 22, 2006 1:06 am
Contact:

Post by SHH »

LOL i just can't win!

Code: Select all

Sitemap Errors
Error 	Detail
Unsupported file format in Sitemap http://pathway2curis.com/forum/forum-1.html 	Your Sitemap does not appear to be in a supported format. Please ensure it meets our Sitemap guidelines and resubmit. More
Unsupported file format in Sitemap http://pathway2curis.com/forum/forum-12.html 	Your Sitemap does not appear to be in a supported format. Please ensure it meets our Sitemap guidelines and resubmit. More
The list goes on, the links work, but google doesn't like them???
FoulFoot
Registered User
Posts: 261
Joined: Mon Jun 23, 2003 1:04 am
Location: Honolulu, HI
Contact:

Re: Hi, just wanted to see if I did everything correctly...

Post by FoulFoot »

Hello --

Haven't been able to get this working on my forum, either. :( To be fair, this is the third phpBB-Google sitemap mod I've tried, and all have failed. Most of this stuff is above my head anyway, so that's probably the main problem.

As a side note, I have about 50 mods installed on my board (not kb or mx), many of which are security mods. I'm guessing that one of them is preventing this mod from functioning -- I already got a "hacking attempt blocked" message when running db_install.php, and needed to do the SQL query manually. I do have the Able2Know SEO mod installed and running fine. Anyway.
dcz wrote: So you should have this folder structure in your phpbb install folder :

mx_ggsitemaps/includes/

with a blank index.html file in it plus two php files (all other do not apply to your case if running phpbb without mx or kb) : index_forum.php and forum_sitemap.php.

I do. Though your install file fails to mention putting the blank .html file in there, and it also neglects to mention the admin/.tpl file. :(
dcz wrote: Then you should have all languages files in phpbb language folders, and admin file in phpbb admin folder and admin tpl in the admin tpl folder.

I do.
dcz wrote: Once this is done, make sure you have set up db properly (this is done by lunching the db instal script)

DB entries are entered, and are correct.

Since I have a main site with phpBB installed in a subfolder, I placed sitemap.php in my root, as you instructed. Lines 40 and 68 were edited. Then, going to www.acaeum.com/sitemap.php in my browser, I get a screen with some random text from my forum, and the error "No sitemaps".
dcz wrote: But you should first make sure it is working in the phpbb folder.

Right, good idea. So I changed lines 40 and 68 back, then uploaded sitemap.php to my forum folder. Accessed it via browser. Got this error message:

Code: Select all

Warning: main(./forum/extension.inc): failed to open stream: No such file or directory in /home/acaeum/public_html/forum/sitemap.php on line 44

Warning: main(): Failed opening './forum/extension.inc' for inclusion (include_path='.:/usr/lib/php:/usr/local/lib/php') in /home/acaeum/public_html/forum/sitemap.php on line 44

Warning: main(./forum/common.): failed to open stream: No such file or directory in /home/acaeum/public_html/forum/sitemap.php on line 45

Warning: main(): Failed opening './forum/common.' for inclusion (include_path='.:/usr/lib/php:/usr/local/lib/php') in /home/acaeum/public_html/forum/sitemap.php on line 45

Fatal error: Call to undefined function: session_pagestart() in /home/acaeum/public_html/forum/sitemap.php on line 51
Which, incidentally, is very similar to the errors I've gotten with other phpBB-Google sitemap mods.

Any insight would be appreciated!

Lastly, for a really stupid question: is this mod actually supposed to create the sitemaps for you? Or just form an index where various sitemaps can be more quickly accessed by Google?

Foul
dcz
Registered User
Posts: 787
Joined: Sun Feb 13, 2005 5:37 am
Contact:

Post by dcz »

@ SHH

Well you misunderstood something here,
I need it to rewrite
http://pathway2curis.com/forum/forum-sitemap-38.xml

to say

http://www.pathway2curis.com/forum/forum-38.html


also does this mod take into account any threads?


No you do not want that, and that's why your sitemap is now invalid.

The first regular sitemap url, is actually the url to the sitemap listing topics from the forum number 38, it's not a link to your forum.
Yesterday, when I loaded this sitemap with the non rewrited url (eg sitemap.php?fid=38 ) I saw it was working perfetctly and told it to you.

Now wen I try to load your sitemap, it seems you uninstalled the mod's table so it does not work any more.

So actually, it was working very well, you just had to move the rewriterules.
## Sitemaps Created :
## - One sitemapindex listing all sitemaps
## - One general forum sitemap, listing public forums url
## - One sitemap per public forum listing topics.
## When kb is installed (both on phpbb and mxBB PORTAL)
## - Adds all necessary entries in the sitemapindex
## - One general categorie sitemap, listing all public categories
## - One sitemap per category, listing approved articles.
## NOTE : Only Cat pointing to a non empty forum and articles with comment will be listed


The sitemap.php is the sitemapIndex, others are real sitemaps.

By the way thanks for the languages mystakes ;)


@ FoulFoot

You need to be logged as admin to lunch the db install script.

Well yes for the blanck html, I will provide two directory strucures in next release, so that there cannot be such problem. I just though the upload of so few files was easy enough, but it was a mystake isn't it :)

The last error comes from the fact you surrely did not edit the sitemap.php file properly while putting it on folder above the phpbb folder.

First try to put it in the same directory as phpbb, have it working, then, you can move it to the root level, making sure you edit the two lines mentionned in the sitemap.php file.

And then, once everything is ok, just submit it.

Fot the able2know mod rewrite, you just need to overwrite the original forum_sitemap.php with the one for able2know in the contrib folder.

Post a link here to make sure everything is working, and trust me, the mod is working ;)

++

phpBB SEO || phpBB3 SEO Premod || SEO phpBB3
GYM Sitemaps & RSS for phpBB3: GYM Sitemaps & RSS
SHH
Registered User
Posts: 52
Joined: Wed Feb 22, 2006 1:06 am
Contact:

Post by SHH »

Hi,

Yes for the time being, i have unistalled it. Will work a little more on it when I'm on spring break. You might want to inlcude this somewhere in the directions. It might have been my problem.

Google has detected some URLs that appear to be at a higher level or different domain than the Sitemap file location. Note that you will not see all denied URLs listed. Rather, Google lists three of them to give you an idea of what types of URLs are being denied. If you receive this error, try moving your Sitemap file to a higher location in your site and submitting again. For instance, if your Sitemap is listed under http://www.example.com/mysite/sitemap.xml, the following URLs are not valid for that sitemap

* http://www.example.com/ - it's at a higher level than the Sitemap
* http://www.example.com/yoursite/ - it's in a directory parallel to the Sitemap
* http://www.example.com/mysite/yoursite - it's missing the trailing slash

You should also check that the urls all begin with the same domain as your Sitemap location. For instance, if your sitemap is listed under http://www.example.com/sitemap.xml, the following URLs are not valid for that sitemap:

* http://www.google.com - it's in the google.com domain rather than the example.com domain
* http://example.com/ - it's missing the initial www
* www.example.com/ - it's missing the protocol (http)
* http://www.example.com - it's missing the trailing slash
* https://www.example.com/ - it's using a different protocol (https rather than http)


So even tho this link was never functional
http://pathway2curis.com/forum/forum-sitemap-38.xml
google still would have recognized it? Maybe i'm not understand the modrewrite rules. "you just had to move the rewriterules"

TIA
dcz
Registered User
Posts: 787
Joined: Sun Feb 13, 2005 5:37 am
Contact:

Post by dcz »

SHH wrote: So even tho this link was never functional
http://pathway2curis.com/forum/forum-sitemap-38.xml
google still would have recognized it? Maybe i'm not understand the modrewrite rules. "you just had to move the rewriterules"

TIA


Not at all actually, and I doubt the url was never functionnal, at least, I have seen http://pathway2curis.com/forum/sitemap.php?fid=38 working great with correct rewrited urls (I swear)

The xml url should have worked when you moved the rewriterules as I adviced you (since sitemaps.xml was, I did not have an opportunity to check the previous though, but I doubt it did not work).

Then as far as google, it's just like I told you, Google only allow url starting from the folder where the sitemap is (or obviously under it)).

Then, if your sitemap.php file (thus all sitemap files) stand in the forum/ folder, then you can list all url starting with : www.example.com/forum/, you cannot for example list this one www.example. com, or this one www.example.com/otherfolder/.
That's why I told how to put the sitemap.php file one level up, so that you can list all of your domain urls, but this is just ment for making it easier for those who whish to use it this way.
You can register many sitemaps for a single domain, so you could also use another sitemap system for the root url, or decide to move this sitemap at a later time, when you feel more confident with it (since you can re register sitemap also, google will not care at all), or none too.

Try to make one thing at a time here, you had done everything well so far and I repeat it was working. There may be a little issue with the rewriterules for forum sitemap (I could not test so) but I doubt it, and even, it would be nothing to fix.

Again, using mod rewrite for sitemaps urls is really a cosmetic feature, google does not care at all about that, all he cares about is finding the right forum and topic url, period.

So you should first make sure the sitemap is working without activating mod rewrite in acp, outputing correct url to your board, and then, knowing it's a cosmetic feature, decide to use it or not.

++

phpBB SEO || phpBB3 SEO Premod || SEO phpBB3
GYM Sitemaps & RSS for phpBB3: GYM Sitemaps & RSS
SHH
Registered User
Posts: 52
Joined: Wed Feb 22, 2006 1:06 am
Contact:

Post by SHH »

Thanks, it seems to be working. with able2knows mod rewrite


I think it is a terminology thing that I was missing... Is this correct?

Here is another question. No sure how to ask this but, all the xml files link you to other xml files until you get down to the .html files. Google will only rank the .html fles correct becuase of this?

Code: Select all

<?xml version='1.0' encoding='UTF-8'?>
<urlset xmlns="http://www.google.com/schemas/sitemap/0.84"
	xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
	xsi:schemaLocation="http://www.google.com/schemas/sitemap/0.84
	http://www.google.com/schemas/sitemap/0.84/sitemap.xsd">
<!--	Generated by mx Google Sitemaps v1.0.0RC1 - (C) 2006 dcz - http://www.phpbb-seo.com/ -->
dcz
Registered User
Posts: 787
Joined: Sun Feb 13, 2005 5:37 am
Contact:

Post by dcz »

Exactly, the sitemap system is just ment to help google finding url in your site, it's not indexed itself.

++

phpBB SEO || phpBB3 SEO Premod || SEO phpBB3
GYM Sitemaps & RSS for phpBB3: GYM Sitemaps & RSS
SHH
Registered User
Posts: 52
Joined: Wed Feb 22, 2006 1:06 am
Contact:

Post by SHH »

Thanks,

Got it working....

:P

now back to quantum chemistry!

Cheers

For anybody who is interested in cancer, how it works, ect you can visit my site, ask questions, or read large number of articles that I have. However, my site deals with specific pathway, but it is a ubiquitous one.

[SPAM]
dcz
Registered User
Posts: 787
Joined: Sun Feb 13, 2005 5:37 am
Contact:

Post by dcz »

SHH wrote: Thanks,

Got it working....

:P

now back to quantum chemistry!

Cheers

For anybody who is interested in cancer, how it works, ect you can visit my site, ask questions, or read large number of articles that I have. However, my site deals with specific pathway, but it is a ubiquitous one.

[SPAM]


You see, you have made it ;)

Hehe and I once too was involved in quantum physics, but the theoritical part, not chemistry. Long time no see Plank and Shrodinger, but, this is not the kind of thing you get rid of ...

++

phpBB SEO || phpBB3 SEO Premod || SEO phpBB3
GYM Sitemaps & RSS for phpBB3: GYM Sitemaps & RSS
SHH
Registered User
Posts: 52
Joined: Wed Feb 22, 2006 1:06 am
Contact:

Post by SHH »

Works like a charm :)

Google has been following phpbb created sitemap links! Google used to just stay on the index but is aggressively crawling (or directed) to many other pages with this mod.


:o
dcz
Registered User
Posts: 787
Joined: Sun Feb 13, 2005 5:37 am
Contact:

Post by dcz »

SHH wrote: Works like a charm :)

Google has been following phpbb created sitemap links! Google used to just stay on the index but is aggressively crawling (or directed) to many other pages with this mod.


:o


Which is the main goal of all this ;)

And yes, spread the world, this mod does help a lot to get indexed, even though we are not talking about page rank improvement here, just being indexed and thus ranked properly.

++

phpBB SEO || phpBB3 SEO Premod || SEO phpBB3
GYM Sitemaps & RSS for phpBB3: GYM Sitemaps & RSS
Post Reply

Return to “[2.0.x] MODs in Development”