[ABD] Google sitemap mod

A place for MOD Authors to post and receive feedback on MODs still in development. No MODs within this forum should be used within a live environment! No new topics are allowed in this forum.
Forum rules
READ: phpBB.com Board-Wide Rules and Regulations

IMPORTANT: MOD Development Forum rules

On February 1, 2009 this forum will be set to read only as part of retiring of phpBB2.
Locked
dcz
Registered User
Posts: 787
Joined: Sun Feb 13, 2005 5:37 am
Contact:

hi

Post by dcz » Sun Jun 19, 2005 10:21 am

@esljunction

When trying to view your sitemap.php I get :

Code: Select all

Erreur d'analyse XML : données bizarres après l'élément de document
Emplacement : http://esljunction.com/jobs/sitemap.php
Numéro de ligne 26721, Colonne 1 :<br />
^
In french, which means : strange datas after documents elements at http://esljunction.com/jobs/sitemap.php line 26721 cell 1 : <br />
^

So for sur it's not you computer, I can see mine but not yours. Maybe there is a ^ somewhere in your code that causing some problem.

++

dcz
Last edited by dcz on Sun Jun 19, 2005 3:54 pm, edited 1 time in total.

phpBB SEO || phpBB3 SEO Premod || SEO phpBB3
GYM Sitemaps & RSS for phpBB3: GYM Sitemaps & RSS

User avatar
esljunction
Registered User
Posts: 27
Joined: Thu May 13, 2004 7:39 pm
Contact:

Post by esljunction » Sun Jun 19, 2005 12:00 pm

Thanks for the reply.

Not really sure what you mean. Are you saying that the sitemap code I installed is incorrect or there is something wrong with the code that comes after sitemap.php make the database query?

i will check all of the updated code now to see if I can find the problem.

Thanks

Patrick Simmons
Registered User
Posts: 296
Joined: Fri May 13, 2005 12:38 pm
Contact:

Post by Patrick Simmons » Mon Jun 20, 2005 3:36 pm

jhaskins wrote: Ok, looking back at the code I posted, one things thaat jumps out at me is that I put the ")" in the wrong place (encoding too much). Grab the code from http://forums.streetrod3.com/sitemap2.phps, that's a copy of a working script.

webmedic - I don't mind. I'm just glad that someone finds it useful :). My reason for using urlencode was a quickfix to account for any special characters that could break the xml. When I release the next version, I'll have better solution in place.


I replaced this with my sitemaps.php and I still can't access my site from any of the url's produced.


Click here to see!

kulinar
Registered User
Posts: 651
Joined: Thu Feb 17, 2005 7:31 pm
Contact:

Post by kulinar » Mon Jun 20, 2005 4:18 pm

Apparently you use GoogleBB Link Enhancement 1.0.0 and webmedic sitemap mod addon at the same time. That is not possible. You should uninstall GoogleBB Link Enhancement 1.0.0, then install webmedic's static keyword URLs mod_rewrite 1.0.0 and then apply webmedic's sitemap mod add-on.

User avatar
webmedic
Registered User
Posts: 323
Joined: Tue May 28, 2002 2:37 am
Contact:

Post by webmedic » Mon Jun 20, 2005 4:51 pm

yes sorry I was not monitoring this thread for posts about my mod since this thread is not about the sitemaps mod I made from it.

R. U. Serious
Registered User
Posts: 830
Joined: Mon Feb 11, 2002 2:07 pm

Post by R. U. Serious » Mon Jun 20, 2005 5:55 pm

Some comments and suggestions:
Although I designed it to keep resource usage as low as possible, I've got no idea how it would handle something that large.


Thanks for you efforts. Additional steps you could take that would improve performance are:

1. Instead of using the templating engine, out put it directly (using echo). The way it currently is, the whole sitemap is generated and assigned to a variable, until it is finished and then that get echoed. You can use output-buffering (with or without ob_gzhandler) and regularly call ob_flush() to prevent having to hold the whole sitemap in memory.
2. Instead of:

Code: Select all

	$topics = $db->sql_fetchrowset($result);
	$db->sql_freeresult();
	foreach ($topics as $topic)
do

Code: Select all

	foreach ($topics as $db->sql_fetchrow($result))
then you can also omit getting the topics in batches of 250. This will be faster overall, and use less memory.
jhaskins wrote: Regarding the number of posts: there is a theoretical limit of 49,999 posts due to the 50,000 url limit on individual sitemaps (this mod also includes an entry the index page).


Actually it's topics, not posts. ;)


Maybe you want to add the links to all public forums as well (with a higher change frequency).

User avatar
webmedic
Registered User
Posts: 323
Joined: Tue May 28, 2002 2:37 am
Contact:

Post by webmedic » Mon Jun 20, 2005 6:20 pm

thank you for the sql optimization.

I can't speak for others but I prefer to use the template engine. It does in the end make it easier if changes need to be made. But of course this is all a matter of opinion. After working many years though on cms systems the end users always prefer templates to separate logic from presentation and I do also.

dcz
Registered User
Posts: 787
Joined: Sun Feb 13, 2005 5:37 am
Contact:

hi

Post by dcz » Wed Jun 22, 2005 2:18 pm

hello,

I am very happy with this mod as google is now for the first time able to spider all my forum, but, as the sitemap.php is located in root/marsforum/, the pages that are eventually in an upper folder such as root/ or /root/otherfolder seem not to be considered by google as part of the site.

Here is what they say about it :
Location of Sitemap Files [Contents]

The location of a Sitemap file determines the set of URLs that can be included in that Sitemap. A Sitemap file located at http://yoursite.com/catalog/sitemap.gz can include any URLs starting with http://yoursite.com/catalog/ but can not include URLs starting with http://yoursite.com/images/.

If you have the permission to change "http://site.org/path/sitemap.gz", it is safe to assume that you also have permission to provide information for URLs with the prefix "http://site.org/path/". Examples of URLs considered valid in http://yoursite.com/catalog/sitemap.gz include:

http://yoursite.com/catalog/show?item=23
http://yoursite.com/catalog/show?item=233&user=3453

URLs not considered valid in http://yoursite.com/catalog/sitemap.gz include:

http://yoursite.com/image/show?item=23
http://yoursite.com/image/show?item=233&user=3453
http://example.com/catalog/show?item=24

URLs that are not considered valid are dropped from further consideration. It is strongly recommended that you place your Sitemap at the root directory of your web server. For example, if your HTTP Web server is at yoursite.com, then your Sitemap index file would be at "http://yoursite.com/sitemap.gz". In certain cases, you may need to produce different Sitemaps for different paths — e.g. if security permissions in your organization compartmentalize write access to different directories.


So, as I hard code in the mod's tpl the rest of my site (not part of my single forum) I was wondering if there is a way to put the sitemap.php file in the root directory so that every links can be taken into consideration by google.

Here is what I get now from them :
URL Error
http://imp.marsatak.org/ URL not under sitemap domain
http://www.marsatak.org/ URL not under sitemap domain


So maybe I won't be able to include -http://imp.marsatak.org/ in the sitemap but it would be cool to include the home page and the other I am going to put online outside the forum.

thanks a lot for this mod anyway.

++

dcz

phpBB SEO || phpBB3 SEO Premod || SEO phpBB3
GYM Sitemaps & RSS for phpBB3: GYM Sitemaps & RSS

User avatar
webmedic
Registered User
Posts: 323
Joined: Tue May 28, 2002 2:37 am
Contact:

Post by webmedic » Wed Jun 22, 2005 3:23 pm

no this is being discussed on the sitemap mailing lists. sitemaps do not support subdomains.

dcz
Registered User
Posts: 787
Joined: Sun Feb 13, 2005 5:37 am
Contact:

hello

Post by dcz » Wed Jun 22, 2005 4:04 pm

thanks for answering webmedic.

There is no problem if this mod cannot handle subdomains, it's just that all the pages located above the forum root are not valid (like www.marsatak.org , the home page I hard coded it in the mod's tpl to have it in the sutemap), and if I understood what google said, the only way to fix that is to put the sitemap.php in the web site's root (on level up for me).

So my sitemap is now in -www.marsatak.org/marsforum/sitemap.php and I would like to put it in -www.marsatak.org/sitemap.php

I know this must be possible, but there must be some code to change too, just to have phpbb knows the file is not in the forum's root dir, but I don't know what to do exaclty.

This would make it possible to use this mod to output the whole sitemap and not only the forum's one.

++

dcz

phpBB SEO || phpBB3 SEO Premod || SEO phpBB3
GYM Sitemaps & RSS for phpBB3: GYM Sitemaps & RSS

kulinar
Registered User
Posts: 651
Joined: Thu Feb 17, 2005 7:31 pm
Contact:

Google sitemap MOD for GoogleBB Link Enhancement 1.0.0

Post by kulinar » Mon Jun 27, 2005 2:02 pm

Google sitemap MOD add-on for GoogleBB Link Enhancement 1.0.0 MOD

Code: Select all

#
#-----[ OPEN ]------------------------------------------
#
sitemap.php

#
#-----[ FIND ]------------------------------------------
#
include($phpbb_root_path . 'common.'.$phpEx);

# 
#-----[ AFTER, ADD ]------------------------------------------ 
#
/**
 * Function: replace_html_links
 *
 * Author: Kinetix < webmaster@ikrontik.tk >
 *
 * Version: 1.0.0
 *
 * Purpose: Transforms page content's links into .html pages
 *   to allow better caching in Google.
 *
 * Notes: This will NOT work properly if you do not use & for
 *   the & character in links.
 */
function replace_html_links($pagedata)
{
	// Whatever you want to replace has to be in the same index as 
	// whatever you are going to replace it with in the changearray
	$replacearray = array(
		"'(?<!/)viewforum.php\?f=([0-9]+)&topicdays=([0-9]+)&start=([0-9]+)'",
		"'(?<!/)viewforum.php\?f=([0-9]+)&(mark|start)=([a-zA-Z0-9_]*)'",
		"'(?<!/)viewforum.php\?f=([0-9]+)'",
		"'(?<!/)viewtopic.php\?t=([0-9]+)&([a-zA-Z]*)=topic&start=([0-9]+)'",
		"'(?<!/)viewtopic.php\?t=([0-9]+)&postdays=([0-9]+)&postorder=([a-zA-Z]*)&vote=([a-zA-Z0-9_]*)'",
		"'(?<!/)viewtopic.php\?t=([0-9]+)&start=([0-9]+)&postdays=([0-9]+)&postorder=([a-zA-Z]*)&highlight='",
		"'(?<!/)viewtopic.php\?t=([0-9]+)&postdays=([0-9]+)&postorder=([a-zA-Z]*)&start=([0-9]+)'",
		"'(?<!/)viewtopic.php\?t=([0-9]+)&(start|view|highlight)=([/:\-\'(){}.&=_a-zA-Z0-9 ]*)'",
		"'(?<!/)viewtopic.php\?t=([0-9]+)&([a-zA-Z0-9]*)=topic&start=0'",
		"'(?<!/)viewtopic.php\?t=([0-9]+)'",
		"'(?<!/)viewtopic.php\?p=([0-9]+)'",
		"'(?<!/)search.php\?([a-zA-Z0-9_]+)=([a-zA-Z0-9_]*)&([a-zA-Z0-9_]*)=([a-zA-Z0-9_]*)'",
		"'(?<!/)search.php\?([a-zA-Z0-9_]+)=([a-zA-Z0-9_]*)'",
		"'(?<!/)search.php'",
		"'(?<!/)memberlist.php\?mode=([a-zA-Z0-9]+)&order=([a-zA-Z]*)&start=([0-9]*)'",
		"'(?<!/)memberlist.php'",
		"'(?<!/)faq.php\?mode=([a-zA-Z0-9_]*)'",
		"'(?<!/)faq.php'",
		"'(?<!/)groupcp.php\?g=([0-9]+)&start=([0-9]+)'",
		"'(?<!/)groupcp.php\?g=([0-9]+)'",
		"'(?<!/)groupcp.php'",
		"'(?<!/)profile.php\?mode=viewprofile&u=([0-9]+)'"
	);
	
	$changearray = array(
		"viewforum.\\1-\\2-\\3.html",
		"viewforum.\\1-\\2\\3.html",
		"viewforum.\\1.html",
		"watchtopic.\\1-\\2-\\3.html",
		"viewtopicres.\\1-\\2-\\3-\\4.html",
		"viewtops.\\1-\\2-\\3.html",
		"viewtopp.\\1-\\2-\\3-\\4.html",
		"viewtoph.\\1-\\2-\\3.html",
		"viewtopx.\\1-\\2.html",
		"viewtopic.\\1\\2.html",
		"viewpost.\\1\\2.html",
		"search.\\1-\\2-\\3-\\4.html",
		"search.\\1-\\2.html",
		"search.html",
		"memberlist.\\1-\\2-\\3.html",
		"memberlist.html",
		"faq.\\1.html",
		"faq.html",
		"groupcp.\\1-\\2.html",
		"groupcp.\\1.html",
		"groupcp.html",
		"viewprofile.\\1.html"
	);
	if(($newdata = preg_replace($replacearray, $changearray, $pagedata)) != FALSE)
		return $newdata;
	else
		return $pagedata;
}

#
#-----[ FIND ]------------------------------------------
#
'TOPIC_URL' => $server_url."viewtopic.$phpEx?t=" . $topic['topic_id'],

# 
#-----[ REPLACE WITH ]------------------------------------------ 
#
//'TOPIC_URL' => $server_url."viewtopic.$phpEx?t=" . $topic['topic_id'],
				'TOPIC_URL' => $server_url.replace_html_links("viewtopic.$phpEx?t=" . $topic['topic_id']),

# 
#-----[ SAVE/CLOSE ALL FILES ]------------------------------------------ 
# 
# EoM 

kulinar
Registered User
Posts: 651
Joined: Thu Feb 17, 2005 7:31 pm
Contact:

Post by kulinar » Tue Jun 28, 2005 8:10 am

Google sitemap MOD add-on for GoogleBB Link Enhancement 1.2.1 MOD

Code: Select all

# 
#-----[ OPEN ]------------------------------------------ 
# 
sitemap.php 

# 
#-----[ FIND ]------------------------------------------ 
# 
include($phpbb_root_path . 'common.'.$phpEx); 

# 
#-----[ AFTER, ADD ]------------------------------------------ 
# 
/** 
 * Function: replace_html_links 
 * 
 * Author: Kinetix < webmaster@ikrontik.tk > 
 */ 
function replace_html_links($content) 
{ 
   $url_in = "'(?<!/)viewtopic.php\?t=([0-9]*)'"; 
   $url_out = "viewtopic.\\1\\2.html"; 

   if (($newdata = preg_replace($url_in, $url_out, $content)) !== false) 
      return $newdata; 
   return $content; 
}

# 
#-----[ FIND ]------------------------------------------ 
# 
'TOPIC_URL' => $server_url."viewtopic.$phpEx?t=" . $topic['topic_id'], 

# 
#-----[ REPLACE WITH ]------------------------------------------ 
# 
//Replacing original code
//'TOPIC_URL' => $server_url."viewtopic.$phpEx?t=" . $topic['topic_id'], 
'TOPIC_URL' => $server_url.replace_html_links("viewtopic.$phpEx?t=" . $topic['topic_id']), 

# 
#-----[ SAVE/CLOSE ALL FILES ]------------------------------------------ 
# 
# EoM
Last edited by kulinar on Wed Aug 10, 2005 1:21 pm, edited 1 time in total.

dcz
Registered User
Posts: 787
Joined: Sun Feb 13, 2005 5:37 am
Contact:

lo

Post by dcz » Tue Jun 28, 2005 10:03 am

hello,

Anybody on how to put the sitemap.php file one directory above in order to take care about what I just posted two and four messages ahead?

Thx!

dcz

phpBB SEO || phpBB3 SEO Premod || SEO phpBB3
GYM Sitemaps & RSS for phpBB3: GYM Sitemaps & RSS

celeb-x
Registered User
Posts: 30
Joined: Sat Jun 18, 2005 10:31 pm
Location: The Netherlands
Contact:

Post by celeb-x » Tue Jun 28, 2005 11:46 am

SQL Error : 1064 You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '' at line 1

SELECT t.topic_id, t.topic_type, t.topic_status, p.post_time FROM forum_topics AS t, forum_posts AS p WHERE t.topic_last_post_id=p.post_id AND t.forum_id IN (1,2,3,4,5,6,7,8,9) ORDER BY t.topic_id LIMIT

Line : 84
File : sitemap.php

http://www.scooterhulp.com/sitemap.php

Anyone knows what the problem is ?

User avatar
webmedic
Registered User
Posts: 323
Joined: Tue May 28, 2002 2:37 am
Contact:

Post by webmedic » Tue Jun 28, 2005 2:38 pm

actually unless you language prohibits you this mod gives better rankings.

and I have modified the sitemap to go along with it and have also removed the parsing errors.

http://www.webmedic.net/released-phpbb- ... 25564.html

http://www.webmedic.net/released-phpbb- ... 25565.html

My sitemap code is based on the code from this mod and the original author gave his ok for me to post it.

Locked

Return to “[2.0.x] MODs in Development”