[ABD] Disable links for new users & guests

[dangerousprototypes]

Couple areas for improvement and further development:

(functionality added, needs ACP support) *Search terms could be entered in a single comma delimited list in the ACP and split apart. That would eliminate the issue of multiple entries you mentioned earlier. Edited:did some research and modified this.

Code: Select all

   //could whitelist other common domains too
   //$no_link_message=str_replace('http://code.google.com', $config['server_name'], $no_link_message);   

(functionality added, needs ACP support) *When we clear our own site, we could also white list friendly sites from an array. On our forum I whitelisted google code where we keep source code, and several other sites that are popular resources to users. A list of about 20 sites covers all the major related sites people might want to link. Other than that it's personal sites which are an unfortunate consequence of the mod, however it is absolute freedom from encumbrance for 99% of users.

*Cron purge - We clear new users who don't post after 48 hours because we get 100s of zombie registrations a day (even with recapthchta, email confirm, and a question). Moderating them (double mod pack) is totally out of the question. It's out of the scope of this mod, but I would like a button or simple field or cron job in teh ACP to clear out dead registrations. I have to check the date to do prune users, it's a pain.

*Actually hide the profile from new users. For extra credit, I'd love to setup an auto-ban for users who post the profile page when it isn't visible (clearly a bot).

*It would be great to only apply this to the new users group, for example. Or somehow exclude regular members if they have been approved.

*For the settings survey - we're using 1 post and 24 hours. Non-posters are purged after 48 hours. They'd have to post two comments in that window to get spam through. We'll see if that happen, and if so we'll auto purge new users after 23 hours instead

You can test this mod here. (is this an acceptable use of linking? am I an author once I revise and post code?) Your account will be auto in 24hours if you don't post.

I have almost no knowledge of php, so if someone wants to adopt this mod, and run with it, feel free. I just think that it is such a useful weapon in the armory against spammers, it will be needed by most of us.

I'd be happy to develop the source further because this is work related. Would someone else be interested in packaging it and doing the HTML the changes to the ACP? I don't know how and it wouldn't be work related.

[dangerousprototypes]

I spent some more time cleaning up the source and adding a few easy features.

Updates:
*set ie $f->load_values_from_db=true; to use the class with the existing mod database values (kinda)
*age and time variables can be set manually as well with:

Code: Select all

	public $load_values_from_db=false; //enable to use with DB, or use the values below
	public $minimum_days=1; //minimum days as member to post links
	public $minimum_posts=1; //minimum posts for member to post links

*A variable to test if it is a new user or not

Code: Select all

	public $filter_user=false; //will tell if we decided to filter user or not

*an array of whitelist urls that are always allowed (in addition to own site)

Code: Select all

	//URLS to ALLOW always. In additon to own-site urls
	public $whitelist_urls=array(); //('http://code.google.com', 'http://sourceforge.net',);

*Currently the soruce tries to load the filter links, words, and whitelist from the database if configured, so it makes it kind of useless unless you add those and insert a value. It could be supported in the ACP, and comma separated lists will be split into an array:

Code: Select all

		$this->minimum_days=$config['links_after_num_days'];
		$this->minimum_posts=$config['links_after_num_posts'];
		$this->whitelist_urls=$config['links_allow_always'];
		$this->no_link_strings=explode(",", $config['links_link_strings']);
		$this->no_word_strings=explode(",", $config['links_word_strings']);		

All-around much cleaner code. Complete untested though.

Code: Select all

<?php
/**
*
* functions_link_filter.php
* @license http://opensource.org/licenses/gpl-license.php GNU Public License
*
*/

/**
* @ignore
*/
if (!defined('IN_PHPBB'))
{
   exit;
}

class link_filter{
	// An array of no-nos. Add whatever you need...
	private $no_link_strings=array('http://', 'www.', '.com', '.us', '.net', '.org', '.uk', '.ly', '.me', '.ru', '.biz', '.info', 'dot com', 'dot net', 'dot org', 'dotcom', 'dotnet', 'dotorg');

	//a secondary spam words filter
	//common words, russian text, etc
	//for unicode blocks see: http://www.fileformat.info/info/unicode/block/index.htm
   private $no_word_strings=array('all natural', 'buy direct', 'gamble', 'gambling',
							"\xD0\x94","\xD0\xB4", //CYRILLIC LETTER DE (A) (capital and lower case)
							"\xD0\x98","\xD0\xB8", //CYRILLIC LETTER I (N)
							"\xD0\x99", "\xD0\xB9", //CYRILLIC LETTER SHORT I (N)
							"\xD0\x9B", "\xD0\xBB", //CYRILLIC LETTER EL (N)
							"\xDA\x9A", "\xDA\xB0", //couple from the arabic block
							//'thanks for sharing',  'new member', 'nice web site', 'nice site',
							);
	
	//URLS to ALLOW always. In additon to own-site urls
	public $whitelist_urls=array(); //('http://code.google.com', 'http://sourceforge.net',);
		
	public $load_values_from_db=false; //enable to use with DB, or use the values below
	public $minimum_days=1; //minimum days as member to post links
	public $minimum_posts=1; //minimum posts for member to post links
	
	public $error=array(); //holds error array
	public $filter_user=false; //will tell if we decided to filter user or not
	

	
/**
*  Test if user can have a profile yet
*  returns true if they can have a profile
*/
function link_filter_test_profile(){
	global $user;
	
	//do we need to check this user?
	if(!$this->link_filter_check()) return false; //don't check, no error
	
	//If there isn't a phpbb3 no_link message add one
	if (empty($user->lang['NO_LINK_FOR_YOU'])){
		$user->lang['NO_PROFILE_FOR_YOU']='Antispam: You can\'t have a profile yet. You need to post a few times first.';
	}
	
	$this->error[]=$user->lang['NO_PROFILE_FOR_YOU'];
	
	return true;
}	

/**
*  Test a submitted signiture for links and words
*  Returns true if bad things detected
*/
function link_filter_test_signiture(){
	global $user;
	
	//do we need to check this user?
	if(!$this->link_filter_check()) return false; //don't check, no error
	
	//If there isn't a phpbb3 no_link message add one
	if (empty($user->lang['NO_LINK_FOR_YOU'])){
		$user->lang['NO_LINK_FOR_YOU']='Antispam: You can\'t have off-site URLs in your sig until you post a few times.';
	}
	//If there isn't a phpbb3 no_word message add one
	if (empty($user->lang['NO_WORD_FOR_YOU'])){
		$user->lang['NO_WORD_FOR_YOU']='Do you kiss your mom with that mouth? We don\'t want to read that!';
	}
	
	//make a version of the post and subject
	//need the trailing space or it can hang forever in the while loop if only using a local URL
	$this->link_filter_test(' '.trim(utf8_normalize_nfc(request_var('signature', (string) $user->data['user_sig'], true))).' ');
	
	if (sizeof($this->error)){
		return true;
	}else{
		return false;
	}
}

/**
*  Test a submitted post for links and words
*  Returns true if bad things detected
*/
function link_filter_test_post(){
	global $user;
	
	//do we need to check this user?
	if(!$this->link_filter_check()) return false; //don't check, no error
	
	//If there isn't a phpbb3 no_link message add one
	if (empty($user->lang['NO_LINK_FOR_YOU'])){
		$user->lang['NO_LINK_FOR_YOU']='I\'m sorry, your post looks too spamy for a new user, try it without off-site URLs.';
	}
	//If there isn't a phpbb3 no_word message add one
	if (empty($user->lang['NO_WORD_FOR_YOU'])){
		$user->lang['NO_WORD_FOR_YOU']='I\'m sorry, your post looks too spamy for a new user, try it without bad words or non-english text.';
	}
	
	//make a version of the post and subject
	$this->link_filter_test(' '.trim(utf8_normalize_nfc(request_var('message', '', true)).' '.utf8_normalize_nfc(request_var('subject', '', true))));

	if (sizeof($this->error)){
		return true;
	}else{
		return false;
	}
}

/**
* 	Do we need to check this user?
* 
*/
function link_filter_check()
{
	global $user;

	if($this->load_values_from_db){ //use MOD setting from database
		$this->minimum_days=$config['links_after_num_days'];
		$this->minimum_posts=$config['links_after_num_posts'];
		$this->whitelist_urls=$config['links_allow_always'];
		$this->no_link_strings=explode(",", $config['links_link_strings']);
		$this->no_word_strings=explode(",", $config['links_word_strings']);		
	}

	//check if the user meets filter criteria
	$this->filter_user=((!$user->data['session_admin']) && (($user->data['user_type']==USER_IGNORE) || ($user->data['user_id']==ANONYMOUS) || ($user->data['user_posts']<=($this->minimum_posts-1)) || ($user->data['user_regdate']>((time())-(86400*$this->minimum_days)))));

	//If you're not special, we filter you
	return $this->filter_user;
}

/**
* 	Search the text for forbidden URLs and text. Add an error to the local error array if found
* 
*/
function link_filter_test($no_link_message){
	global $user, $config;

	//filter the looozers
	//remove line feeds and stuff
	$no_link_message=str_replace('\n', ' ',$no_link_message);
	$no_link_message=str_replace('\r', ' ', $no_link_message);
	//replace double spaces with single spaces (not sure why, white space?)
	while (strpos($no_link_message, '  ')){
		$no_link_message=str_replace('  ', ' ', $no_link_message);
	}
	
	//remove any own-site references, these are ok
	//first change http://mysite.com to mysite.com so we only have to look once below
	$no_link_message=str_replace($config['server_protocol'].$config['server_name'], $config['server_name'], $no_link_message);

	//whitelist other common domains too
	//we do this by relacing them with our own domain so we only have to run the search once below
	for ($x=0;$x<sizeof($this->whitelist_urls);$x++){
		if(stripos($no_link_message, $this->whitelist_urls[$x])){
			$no_link_message=str_ireplace($this->whitelist_urls[$x], $config['server_name'], $no_link_message);	
		}
	}
	
	//look at all instances of mysite.com
	while ($ok_start=stripos($no_link_message, $config['server_name'])){ //start of mysite.com
		$ok_end=strpos($no_link_message, '[', $ok_start); //find next [ (bbcode?)
		if (!$ok_end){ //if not bbcode
			$ok_end=strpos($no_link_message, ' ', $ok_start); //end is position of next space
		}
		if ($ok_end){
			$no_link_message=substr($no_link_message, 0, $ok_start).substr($no_link_message, $ok_end);//remove own URL
		}
	}
		
	//search for each link element, throw an error when found
	for ($x=0;$x<sizeof($this->no_link_strings);$x++){
		if (stripos($no_link_message, $this->no_link_strings[$x])){
			$this->error[]=$user->lang['NO_LINK_FOR_YOU'];
			//$x=sizeof($no_link_strings);
			break;//no reason to go further
		}
	}
	
	//search each word, throw an error when found
	for ($x=0;$x<sizeof($this->no_word_strings);$x++){
		if (stripos($no_link_message, $this->no_word_strings[$x])){
			$this->error[]=$user->lang['NO_WORD_FOR_YOU'];
			//$x=sizeof($no_link_strings);
			break;//no reason to search further
		}
	}

}

}//class

?>

[dangerousprototypes]

Further cleanup of the class:
*optimized to only setup the arrays if we are going to filter the user
*cleaner structure and returns
*store results in accessible variables
*Option to limit new user group only (untested)
*option to show the word that triggered the filter so the user isn't left hanging

Code: Select all

<?php
/**
*
* functions_link_filter.php
* @license http://opensource.org/licenses/gpl-license.php GNU Public License
*
*/

/**
* @ignore
*/
if (!defined('IN_PHPBB'))
{
   exit;
}

class link_filter{

	//--settings variables--//
	// An array of no-nos. Add whatever you need...
	private $no_link_strings=array('http://', 'www.', '.com', '.us', '.net', '.org', '.uk', '.ly', '.me', '.ru', '.biz', '.info', 'dot com', 'dot net', 'dot org', 'dotcom', 'dotnet', 'dotorg');

	//a secondary spam words filter
	//common words, russian text, etc
	//for unicode blocks see: http://www.fileformat.info/info/unicode/block/index.htm
   private $no_word_strings=array('all natural', 'buy direct', 'home business', 'premium', 
							"\xD0\x94","\xD0\xB4", //CYRILLIC LETTER DE (A) (capital and lower case)
							"\xD0\x98","\xD0\xB8", //CYRILLIC LETTER I (N)
							"\xD0\x99", "\xD0\xB9", //CYRILLIC LETTER SHORT I (N)
							"\xD0\x9B", "\xD0\xBB", //CYRILLIC LETTER EL (N)
							"\xDA\x9A", "\xDA\xB0", //couple from the arabic block
							//'thanks for sharing',  'new member', 'nice web site', 'nice site',
							);
	
	//URLS to ALLOW always. In additon to own-site urls
	public $whitelist_urls=array(); //('http://code.google.com', 'http://sourceforge.net',);
	
	public $show_trigger_word=true; //show the user the word that triggered the error
		
	public $load_values_from_db=false; //enable to use with DB, or use the values below
	public $minimum_days=1; //minimum days as member to post links
	public $minimum_posts=1; //minimum posts for member to post links
	
	//-- Reporting variables--//
	
	public $error=array(); //holds text error array
	
	public $filter_user=false; //we decided to filter this user (they met our criteria)
	public $found_stuff=false; //did we find anything?
	public $found_links=false; //we found links
	public $found_words=false;//we found bad works

	
/**
*  Test if user can have a profile yet
*  returns false if they do NOT need to be filtered
*/
function link_filter_test_profile(){
	global $user;
	
	//do we need to check this user?
	if(!$this->link_filter_check()) return false; //don't check, no error
	
	if($this->load_values_from_db) $this->link_filter_load_list_from_db(); //load list values only if needed, saves cycles
	
	//If there isn't a phpbb3 no_link message add one
	if (empty($user->lang['NO_LINK_FOR_YOU'])){
		$user->lang['NO_PROFILE_FOR_YOU']='Antispam: You can\'t have a profile yet. You need to post a few times first.';
	}
	
	$this->error[]=$user->lang['NO_PROFILE_FOR_YOU'];
	
	return true;
}	

/**
*  Test a submitted signiture for links and words
*  Returns true if bad things detected
*/
function link_filter_test_signiture(){
	global $user;
	
	//do we need to check this user?
	if(!$this->link_filter_check()) return false; //don't check, no error
	
	if($this->load_values_from_db) $this->link_filter_load_list_from_db(); //load list values only if needed, saves cycles
	
	//If there isn't a phpbb3 no_link message add one
	if (empty($user->lang['NO_LINK_FOR_YOU'])){
		$user->lang['NO_LINK_FOR_YOU']='Antispam: You can\'t have off-site URLs in your sig until you post a few times.';
	}
	//If there isn't a phpbb3 no_word message add one
	if (empty($user->lang['NO_WORD_FOR_YOU'])){
		$user->lang['NO_WORD_FOR_YOU']='Do you kiss your mom with that mouth? We don\'t want to read that!';
	}
	
	//make a version of the post and subject
	//need the trailing space or it can hang forever in the while loop if only using a local URL
	return $this->link_filter_test(' '.trim(utf8_normalize_nfc(request_var('signature', (string) $user->data['user_sig'], true))).' ');
	
}

/**
*  Test a submitted post for links and words
*  Returns true if bad things detected
*/
function link_filter_test_post(){
	global $user;
	
	//do we need to check this user?
	if(!$this->link_filter_check()) return false; //don't check, no error
	
	if($this->load_values_from_db) $this->link_filter_load_list_from_db(); //load list values only if needed, saves cycles
	
	//If there isn't a phpbb3 no_link message add one
	if (empty($user->lang['NO_LINK_FOR_YOU'])){
		$user->lang['NO_LINK_FOR_YOU']='Your post looks too spamy for a new user, please remove off-site URLs.';
	}
	//If there isn't a phpbb3 no_word message add one
	if (empty($user->lang['NO_WORD_FOR_YOU'])){
		$user->lang['NO_WORD_FOR_YOU']='Your post looks too spamy for a new user, please remove bad words or non-english text.';
	}
	
	//make a version of the post and subject
	return $this->link_filter_test(' '.trim(utf8_normalize_nfc(request_var('message', '', true)).' '.utf8_normalize_nfc(request_var('subject', '', true))));
}

/**
* 	Do we need to check this user?
* 
*/
function link_filter_check()
{
	global $user;

	if($this->load_values_from_db){ //use MOD setting from database
		$this->minimum_days=$config['links_after_num_days'];
		$this->minimum_posts=$config['links_after_num_posts'];
	}

	//check if the user meets filter criteria
	$this->filter_user=((!$user->data['session_admin']) && (($user->data['user_type']==USER_IGNORE) || ($user->data['user_id']==ANONYMOUS) || ($user->data['user_posts']<=($this->minimum_posts-1)) || ($user->data['user_regdate']>((time())-(86400*$this->minimum_days)))));

	//this MIGHT be used in a bigger filter to only apply to new users.
	//$this->filter_user=((!$user->data['session_admin']) && (($user->data['user_type']==USER_IGNORE) || ($user->data['user_id']==ANONYMOUS) || ($user->data['user_new']==1 &&(($user->data['user_posts']<=($this->minimum_posts-1)) || ($user->data['user_regdate']>((time())-(86400*$this->minimum_days)))))));

	//If you're not special, we filter you
	return $this->filter_user;
}

/**
* 	if database is enabled this will explore the filter lists into the arrays
*  We move it here so this only happen when we need to filter the user
*/
function link_filter_load_list_from_db(){
	//use MOD setting from database
	$this->whitelist_urls=explode(",", $config['links_allow_always']);
	$this->no_link_strings=explode(",", $config['links_link_strings']);
	$this->no_word_strings=explode(",", $config['links_word_strings']);		
}

/**
* 	Search the text for forbidden URLs and text. 
*	Add an error to the local error array if found
*	returns true for bad stuff, false for no flags found
* 
*/
function link_filter_test($no_link_message){
	global $user, $config;

	//filter the looozers
	//remove line feeds and stuff
	$no_link_message=str_replace('\n', ' ',$no_link_message);
	$no_link_message=str_replace('\r', ' ', $no_link_message);
	//replace double spaces with single spaces (not sure why, white space?)
	while (strpos($no_link_message, '  ')){
		$no_link_message=str_replace('  ', ' ', $no_link_message);
	}
	
	//remove any own-site references, these are ok
	//first change http://mysite.com to mysite.com so we only have to look once below
	$no_link_message=str_replace($config['server_protocol'].$config['server_name'], $config['server_name'], $no_link_message);

	//whitelist other common domains too
	//we do this by relacing them with our own domain so we only have to run the search once below
	for ($x=0;$x<sizeof($this->whitelist_urls);$x++){
		if(stripos($no_link_message, $this->whitelist_urls[$x])){
			$no_link_message=str_ireplace($this->whitelist_urls[$x], $config['server_name'], $no_link_message);	
		}
	}
	
	//look at all instances of mysite.com
	while ($ok_start=stripos($no_link_message, $config['server_name'])){ //start of mysite.com
		$ok_end=strpos($no_link_message, '[', $ok_start); //find next [ (bbcode?)
		if (!$ok_end){ //if not bbcode
			$ok_end=strpos($no_link_message, ' ', $ok_start); //end is position of next space
		}
		if ($ok_end){
			$no_link_message=substr($no_link_message, 0, $ok_start).substr($no_link_message, $ok_end);//remove own URL
		}
	}
		

	//search for each link element, throw an error when found
	for ($x=0;$x<sizeof($this->no_link_strings);$x++){
		if (stripos($no_link_message, $this->no_link_strings[$x])){
			
			$this->found_links=true;
						
			$this->error[]=$user->lang['NO_LINK_FOR_YOU'];
			
			//$x=sizeof($no_link_strings);
			break;//no reason to go further
		}
	}
	
	//search each word, throw an error when found
	for ($x=0;$x<sizeof($this->no_word_strings);$x++){
		if (stripos($no_link_message, $this->no_word_strings[$x])){
		
			$this->found_words=true;
		
			if($this->show_trigger_word){//show the cause so the user isn't stumped 
				$this->error[]=$user->lang['NO_WORD_FOR_YOU'].' ('.$this->no_word_strings[$x].')';
			}else{//don't show the cause
				$this->error[]=$user->lang['NO_WORD_FOR_YOU'];
			}
			
			//$x=sizeof($no_link_strings);
			break;//no reason to search further
		}
	}
	
	if($this->found_links || $this->found_words)$this->found_stuff=true;

	return $this->found_stuff;

}

}//class

?>

[Philthy]

Some nice code there.
I have my new users go put in the new user group, and disallow signatures in that group, but your method seems much simpler, and lets the user know what is happening.

[Philthy]

I've been doing some testing on your site.
Very nice, it picks up just about everything ? Everything I could think of at least.
I have made ten posts though, and it is still restricting me. It appears that it is applying the filters to all users, or is it just newbies? I also noticed that the whitelist didn't seem to work?

If we can get the array of link nono's (.com, .org, .ru, http, www, etc) configurable in the admin panel, along with an array of bad words, also configurable in the admin panel, this could be a very powerful anti spam mod.

Can I suggest a wish list?
In admin panel:
Configure number of posts before allowed to post links/bad words.
Configure number of days membership before allowed to post links/bad words.
Configure array of domain references (.com .org www etc).
Configure bad words (viagra cialis sex etc).

Dangerousprototypes additions block links/bad words in posts/profile/signature, and are a big step forward towards what could be an extremely useful tool for admins.

Nice work DP

[Philthy]

To the MODS:
I'm not sure what to title this mod [RC]or [DEV]? The basic mod I'd created, stands on its own. Perhaps dangerousprototypes should start his own mod in development?

[dangerousprototypes]

Very nice, it picks up just about everything ? Everything I could think of at least.

I should maybe look into the scum issue you encountered with the filter, but it would probably be a lot of work for little payoff. I prefer to tell the user what is filtered so they can fix it. Bot scripts don't care, and people-spammers will give up because they can't post links.

I have made ten posts though, and it is still restricting me.

Yup, it is posts and time. Usually we just get single ego spams (great site, thanks for posting!), but often we also get spray spam in a bunch of threads (to go through the notification system I'd guess). This prevents one lame post followed by tons of spam. The forum is user support for a business, so someone usually checks it at least every 3 hours (usually constantly), giving us enough time to catch anything before they come back in X days and spam.

It appears that it is applying the filters to all users, or is it just newbies?

It is the same as your mod, just teased into a class and applied in more places. It chooses who to filter by posts AND time, no matter the rank (unless admin). There is a commented out line in the class that *should* limit this to the new users group. I have not tested though.

I also noticed that the whitelist didn't seem to work?

It should work with own-site, but I forgot to add back in my external whitelist after the last update.

If we can get the array of link nono's (.com, .org, .ru, http, www, etc) configurable in the admin panel, along with an array of bad words, also configurable in the admin panel, this could be a very powerful anti spam mod.

Can I suggest a wish list?
In admin panel:
....

This could be done, as far as I can tell, exactly the same way you added the other stuff to teh ACP. Add a database entry, and add a place to enter the value in the ACP. I only know what I read in the install.xml for this mod though, I might way underestimate the work involved

The class is already configured to accept the existing posts and days values from your mod:

Code: Select all

   if($this->load_values_from_db){ //use MOD setting from database
      $this->minimum_days=$config['links_after_num_days'];
      $this->minimum_posts=$config['links_after_num_posts'];

and will take a string of domains, whitelists, and words from fields called:

Code: Select all

   //use MOD setting from database
   $this->whitelist_urls=explode(",", $config['links_allow_always']);
   $this->no_link_strings=explode(",", $config['links_link_strings']);
   $this->no_word_strings=explode(",", $config['links_word_strings']);    

It should work right off the bat. Just set $f->load_values_from_db=true; after the $f= new link_filter (or edit to be the default before uploading).

Perhaps dangerousprototypes should start his own mod in development?

I'd really prefer to contribute to this one. The code is 99% from this mod. Can we work together? I'm having fun working on the back end, but I probably won't package it up and add admin panel stuff myself. I'll even commit to maintaining it through future versions (since it is work related).
-----
Another option available is to show the filter word to the user or not, this can eliminate frustration.

I'm going to add a few other features:
*PMs? never had a problem with PM spam, but it is only a matter of time
*auto-ban/delete sleepers. If lots of forums use this approach spammers will create an account, age it until they can post links, then blast a ton through at once. At least that is what I would do, and it is totally scriptable. To prevent this, I already delete new users who are inactive 24 hours after registering. I do this with some ugly sql on a cron job or the prune users menu. I'd like to add this type of function to the mod in three ways:
1- The mod can easily deny access to any user with 0 posts who is outside the age window. It will give a message like "Anitspam: Posting from this account have been disabled, please contact an admin" instead of the other messages. Will do this ASAP.
2 - A little more aggressive - if a new user (0 posts) comes back AFTER the age period and tries to post they will be banned or outright deleted and told to re-register. This is a little more difficult because I have to dig in the api and figure out the right way to do this.
3 - Zombie cleanup - an api-appropriate way to mass delete users older than the minimum post time who have never posted. This cleans up the robot registrations who never post, and prevents "aging" an account to use it for spam later.
*Selectable character set detection, to mass filter unicode characters.

[_Al]

Thanks for your work

[Philthy]

Work together
My script knowledge is almost non existent, but yeah, perfectly happy. I cobbled together the original from pieces put together by other phpbb mod writers. I can't even claim to have had the original idea, as it came from a mod designed for ppbb2. It struck me as the simplest way of kicking spammers though. For the main, their actions are predictable, and blocking external links for just a short while, is a very minor inconvenience to 99.99% of genuine forum users. Adding spam words is the icing on the cake. An auto ban function may just be the cherry on top !

Next question for any script writers out there then, is "how to list an array of domain references, and spam words into the admin, via the database?

[dangerousprototypes]

Thanks!

Next question for any script writers out there then, is "how to list an array of domain references, and spam words into the admin, via the database?

I think the existing code can be recycled. Just create new database entries, then add a field to the ACP. Enter the values as a plain text, comma separated list:

spam1, spam2, spam3,

http://,.com,.org,

I don;t think you need to worry about the array part. The explode(",",...) command takes the string of text and splits it into an array where there are commas. Another character (;) could be used too, just let me know what you decide (currently set for commas).

This will work for everything but the unicode detection, which has to be processed as an actual HEX value. I'll have to think about that a little. My bigger goal is to find a php command that just detects certain unicode groups, then we can test by set (arabic, cryllic, etc) instead of specific characters.

[Philthy]

You see, I'm out of my depth already

Here is the sql that creates the links after num posts/days. I have no idea what needs adding to create an array/string of domain references/bad words?

Code: Select all

INSERT INTO phpbb_config (config_name, config_value, is_dynamic) VALUES ('links_after_num_posts', '0', '0');
INSERT INTO phpbb_config (config_name, config_value, is_dynamic) VALUES ('links_after_num_days', '0', '0');

I'm guessing something like ?

Code: Select all

INSERT INTO phpbb_config (config_name, config_value, is_dynamic) STRING ('links_word_strings', 'viagra', 'cialis', 'sex');

As I said, my script knowledge is extremely limited.

[dangerousprototypes]

No worries, I can walk you through the php and SQL.

I'm guessing this will do it for the new values, the thing I'm not sure on is if config_value will take strings or only numbers, but I'm assuming its variable characters (varchar in phpmyadmin) and will work.

Code: Select all

INSERT INTO phpbb_config (config_name, config_value, is_dynamic) VALUES ('links_allow_always', '', '0');
INSERT INTO phpbb_config (config_name, config_value, is_dynamic) VALUES ('links_link_strings', 'http://,.com,.org,.ru,.us', '0');
INSERT INTO phpbb_config (config_name, config_value, is_dynamic) VALUES ('links_word_strings', 'home mortage,forex,free games,viagra', '0');

Something like that. I can give you my full filter list, it is based on actual spam IO kept from a 30 day period, but it is pretty nasty

For the UTF8 stuff, I can do something like this that counts the total utf8 characters in a random sample and then calculates a ratio or plain to utf8. That way a forum with a few utf8 characters (.de? .es?) can still set a 50-80% threshold and avoid russian language spam:
http://www.mawhorter.net/web-developmen ... -languages

Another option I noted was searching individual ranges for different sets. We could set a bunch and make it an option, but I think it is better (cycle wise) if the user just enters a custom regex when required (examples make it not too difficult).

[dangerousprototypes]

Code: Select all

[\u0400-\u04FF]+

This seems to be the magic regex for the cyrillic utf8 group. It's just a matter of finding the ranges to exclude for the characters you'll never use, and figuring out how to chain them together if not contiguous (| in between I think).

I think the previous method (% utf8) will work best for my forum (and english forums), but won't help a Japanese forum with Russian spam, for example. The method here with the custom ranges would help them.

[dangerousprototypes]

Using this:

Code: Select all

if(preg_match('/([\u0400-\u04FF]|[\u0500-\u05FF])+u/', $str, $m)==1){ //add s if newline, but we already pulled it out (match one character from any of the ranges, treat as utf-8
	//illegal character
}

And these unicode guides:
http://www.fileformat.info/info/unicode/block/index.htm

I think it should be possible to detect the bad alphabets without entering individual characters in to the bad words search. It will also be much more robust.

Any regex gurus out there to point out where I went terribly wrong I'll do some testing later.

Here's a futher updated (not tested) version that displays an error to "sleeper agent" users who never post and then try to post after the defined period (probably won't work if you have a 0 day limit set). For example if you have 0 posts and come back after the 1 day link limit and start to spam, this will prohibit posting and give the warning:

'Antispam: account disabled, please contact an admin.'

I peeked at the ban/delete/purge stuff in the API documentation but it was unhelpful, I'll need to see actual code to figure out if that is feasible.

Code: Select all

<?php
/**
*
* functions_link_filter.php
* @license http://opensource.org/licenses/gpl-license.php GNU Public License
*
*/

/**
* @ignore
*/
if (!defined('IN_PHPBB'))
{
   exit;
}

class link_filter{

	//--settings variables--//
	// An array of no-nos. Add whatever you need...
	private $no_link_strings=array('http://', 'www.', '.com', '.us', '.net', '.org', '.uk', '.ly', '.me', '.ru', '.biz', '.info', 'dot com', 'dot net', 'dot org', 'dotcom', 'dotnet', 'dotorg');

	//a secondary spam words filter
	//common words, russian text, etc
	//for unicode blocks see: http://www.fileformat.info/info/unicode/block/index.htm
   private $no_word_strings=array('players', 'gamble', 'gambling',
							"\xD0\x94","\xD0\xB4", //CYRILLIC LETTER DE (A) (capital and lower case)
							"\xD0\x98","\xD0\xB8", //CYRILLIC LETTER I (N)
							"\xD0\x99", "\xD0\xB9", //CYRILLIC LETTER SHORT I (N)
							"\xD0\x9B", "\xD0\xBB", //CYRILLIC LETTER EL (N)
							"\xDA\x9A", "\xDA\xB0", //couple from the arabic block
							//'thanks for sharing',  'new member', 'nice web site', 'nice site',
							);
	
	//URLS to ALLOW always. In additon to own-site urls
	public $whitelist_urls=array(); //('http://code.google.com', 'http://sourceforge.net',);
	
	public $show_trigger_word=true; //show the user the word that triggered the error
		
	public $load_values_from_db=false; //enable to use with DB, or use the values below
	public $minimum_days=1; //minimum days as member to post links
	public $minimum_posts=1; //minimum posts for member to post links
	
	public $sleeper_check=true; //users with 0 posts who try to post after minimum_days will be prohibited
	//-- Reporting variables--//
	public $filter_user=false; //we decided to filter this user (they met our criteria)
	
	public $found_stuff=false; //did we find anything? (any item below)
	public $sleeper_agent=false;//did we determine this user to be a sleeper agent?
	public $found_links=false; //we found links
	public $found_words=false;//we found bad words
	
	public $error=array(); //holds text error array

	
/**
*  Test if user can have a profile yet
*  returns false if they do NOT need to be filtered
*/
function link_filter_test_profile(){
	global $user;
	
	//do we need to check this user?
	if(!$this->link_filter_check()) return false; //don't check, no error
	
	if($this->load_values_from_db) $this->link_filter_load_list_from_db(); //load list values only if needed, saves cycles
	
	//If there isn't a phpbb3 no_link message add one
	if (empty($user->lang['NO_LINK_FOR_YOU'])){
		$user->lang['NO_PROFILE_FOR_YOU']='Antispam: You can\'t have a profile yet. You need to post a few times first.';
	}
	
	$this->error[]=$user->lang['NO_PROFILE_FOR_YOU'];
	
	return true;
}	

/**
*  Test a submitted signiture for links and words
*  Returns true if bad things detected
*/
function link_filter_test_signiture(){
	global $user;
	
	//do we need to check this user?
	if(!$this->link_filter_check()) return false; //don't check, no error
	
	if($this->load_values_from_db) $this->link_filter_load_list_from_db(); //load list values only if needed, saves cycles
	
	//If there isn't a phpbb3 no_link message add one
	if (empty($user->lang['NO_LINK_FOR_YOU'])){
		$user->lang['NO_LINK_FOR_YOU']='Antispam: You can\'t have off-site URLs in your sig until you post a few times.';
	}
	//If there isn't a phpbb3 no_word message add one
	if (empty($user->lang['NO_WORD_FOR_YOU'])){
		$user->lang['NO_WORD_FOR_YOU']='Do you kiss your mom with that mouth? We don\'t want to read that!';
	}
	
	//make a version of the post and subject
	//need the trailing space or it can hang forever in the while loop if only using a local URL
	return $this->link_filter_test(' '.trim(utf8_normalize_nfc(request_var('signature', (string) $user->data['user_sig'], true))).' ');
	
}

/**
*  Test a submitted post for links and words
*  Returns true if bad things detected
*/
function link_filter_test_post(){
	global $user;
	
	//do we need to check this user?
	if(!$this->link_filter_check()) return false; //don't check, no error
	
	if($this->load_values_from_db) $this->link_filter_load_list_from_db(); //load list values only if needed, saves cycles
	
	//If there isn't a phpbb3 no_link message add one
	if (empty($user->lang['NO_LINK_FOR_YOU'])){
		$user->lang['NO_LINK_FOR_YOU']='Your post looks too spamy for a new user, please remove off-site URLs.';
	}
	//If there isn't a phpbb3 no_word message add one
	if (empty($user->lang['NO_WORD_FOR_YOU'])){
		$user->lang['NO_WORD_FOR_YOU']='Your post looks too spamy for a new user, please remove bad words or non-english text.';
	}
	
	//make a version of the post and subject
	return $this->link_filter_test(' '.trim(utf8_normalize_nfc(request_var('message', '', true)).' '.utf8_normalize_nfc(request_var('subject', '', true))));
}

/**
* 	Do we need to check this user?
* 
*/
function link_filter_check()
{
	global $user;

	if($this->load_values_from_db){ //use MOD setting from database
		$this->minimum_days=$config['links_after_num_days'];
		$this->minimum_posts=$config['links_after_num_posts'];
	}

	//check if the user meets filter criteria
	$this->filter_user=((!$user->data['session_admin']) && (($user->data['user_type']==USER_IGNORE) || ($user->data['user_id']==ANONYMOUS) || ($user->data['user_posts']<=($this->minimum_posts-1)) || ($user->data['user_regdate']>((time())-(86400*$this->minimum_days)))));

	//this MIGHT be used in a bigger filter to only apply to new users.
	//$this->filter_user=((!$user->data['session_admin']) && (($user->data['user_type']==USER_IGNORE) || ($user->data['user_id']==ANONYMOUS) || ($user->data['user_new']==1 &&(($user->data['user_posts']<=($this->minimum_posts-1)) || ($user->data['user_regdate']>((time())-(86400*$this->minimum_days)))))));

	//check for spammers who come back later and  try to post (sleeper agents, or aged accounts)
	if(($this->sleeper_check) && ($user->data['user_posts']==0) && ($user->data['user_regdate']<=((time())-(86400*$this->minimum_days)))){
		$this->sleeper_agent=true;
		
		//If there isn't a phpbb3 sleeper agent message add one
		if (empty($user->lang['NO_SLEEPER_SPAM_FOR_YOU'])){
			$user->lang['NO_SLEEPER_SPAM_FOR_YOU']='Antispam: account disabled, please contact an admin.';
		}
		$this->error[]=$user->lang['NO_SLEEPER_SPAM_FOR_YOU'];
	}
	
	//If you're not special, we filter you
	return $this->filter_user;
}

/**
* 	if database is enabled this will explore the filter lists into the arrays
*  We move it here so this only happen when we need to filter the user
*/
function link_filter_load_list_from_db(){
	//use MOD setting from database
	$this->whitelist_urls=explode(",", $config['links_allow_always']);
	$this->no_link_strings=explode(",", $config['links_link_strings']);
	$this->no_word_strings=explode(",", $config['links_word_strings']);		
}

/**
* 	Search the text for forbidden URLs and text. 
*	Add an error to the local error array if found
*	returns true for bad stuff, false for no flags found
* 
*/
function link_filter_test($no_link_message){
	global $user, $config;

	//filter the looozers
	//remove line feeds and stuff
	$no_link_message=str_replace('\n', ' ',$no_link_message);
	$no_link_message=str_replace('\r', ' ', $no_link_message);
	//replace double spaces with single spaces (not sure why, white space?)
	while (strpos($no_link_message, '  ')){
		$no_link_message=str_replace('  ', ' ', $no_link_message);
	}
	
	//remove any own-site references, these are ok
	//first change http://mysite.com to mysite.com so we only have to look once below
	$no_link_message=str_replace($config['server_protocol'].$config['server_name'], $config['server_name'], $no_link_message);

	//whitelist other common domains too
	//we do this by relacing them with our own domain so we only have to run the search once below
	for ($x=0;$x<sizeof($this->whitelist_urls);$x++){
		if(stripos($no_link_message, $this->whitelist_urls[$x])){
			$no_link_message=str_ireplace($this->whitelist_urls[$x], $config['server_name'], $no_link_message);	
		}
	}
	
	//look at all instances of mysite.com
	while ($ok_start=stripos($no_link_message, $config['server_name'])){ //start of mysite.com
		$ok_end=strpos($no_link_message, '[', $ok_start); //find next [ (bbcode?)
		if (!$ok_end){ //if not bbcode
			$ok_end=strpos($no_link_message, ' ', $ok_start); //end is position of next space
		}
		if ($ok_end){
			$no_link_message=substr($no_link_message, 0, $ok_start).substr($no_link_message, $ok_end);//remove own URL
		}
	}
		

	//search for each link element, throw an error when found
	for ($x=0;$x<sizeof($this->no_link_strings);$x++){
		if (stripos($no_link_message, $this->no_link_strings[$x])){
			
			$this->found_links=true;
						
			$this->error[]=$user->lang['NO_LINK_FOR_YOU'];
			
			//$x=sizeof($no_link_strings);
			break;//no reason to go further
		}
	}
	
	//search each word, throw an error when found
	for ($x=0;$x<sizeof($this->no_word_strings);$x++){
		if (stripos($no_link_message, $this->no_word_strings[$x])){
		
			$this->found_words=true;
		
			if($this->show_trigger_word){//show the cause so the user isn't stumped 
				$this->error[]=$user->lang['NO_WORD_FOR_YOU'].' ('.$this->no_word_strings[$x].')';
			}else{//don't show the cause
				$this->error[]=$user->lang['NO_WORD_FOR_YOU'];
			}
			
			//$x=sizeof($no_link_strings);
			break;//no reason to search further
		}
	}
	
	/* Future UTF8 stuff
	
	
	
	
	
	*/
	
	
	if($this->found_links || $this->found_words ||$this->sleeper_agent) $this->found_stuff=true;

	return $this->found_stuff;

}

}//class

?>

[dangerousprototypes]

In progress update. Includes these new features:
*Minor bug fixes
*can search a range or all unicode characters and reject posts separately from the word check
*if you enter a help link (maybe a post in your forum) it will show it when it displays an error. This can let users know the conditions of the restrictions in your forum
*added a function to delete all users older than the age limit who have 0 posts. Based on the prune code of the ACP. not actually implemented anywhere (no way to run it without a button in the ACP). Uses retain posts delete (they shouldn't have any anyways) in case something goes sour.
*minor efficiency update

This is largely untested, but I'm posting it in case anyone is following.

This will probably be my final feature addition. We could easily add an auto-delete feature, but it seems redundant with the sleeper agent detection and regular zombie account purges.

Next I plan to do a few more things:
*full testing
*Figure out the database and add ACP fields
*Document the features and options
*Add checks to the PM system?
*Anything else? any suggestions?

Code: Select all

<?php
/**
*
* functions_link_filter.php
* @license http://opensource.org/licenses/gpl-license.php GNU Public License
*
*/

/**
* @ignore
*/
if (!defined('IN_PHPBB'))
{
   exit;
}

class link_filter{

	//--settings variables--//
	// An array of no-nos. Add whatever you need...
	private $no_link_strings=array('http://', 'www.', '.com', '.us', '.net', '.org', '.uk', '.ly', '.me', '.ru', '.biz', '.info', 'dot com', 'dot net', 'dot org', 'dotcom', 'dotnet', 'dotorg');

	//a secondary spam words filter
	//common words, russian text, etc
	//for unicode blocks see: http://www.fileformat.info/info/unicode/block/index.htm
   private $no_word_strings=array('all natural', 'buy direct', 'home business', 'premium', 
							"\xD0\x94","\xD0\xB4", //CYRILLIC LETTER DE (A) (capital and lower case)
							"\xD0\x98","\xD0\xB8", //CYRILLIC LETTER I (N)
							"\xD0\x99", "\xD0\xB9", //CYRILLIC LETTER SHORT I (N)
							"\xD0\x9B", "\xD0\xBB", //CYRILLIC LETTER EL (N)
							"\xDA\x9A", "\xDA\xB0", //couple from the arabic block
							//'thanks for sharing',  'new member', 'nice web site', 'nice site',
							);
	
	//URLS to ALLOW always. In additon to own-site urls
	private $whitelist_urls=array(); //('http://code.google.com', 'http://sourceforge.net',);
	
	//a regex to filter a unicode character range. Will throw an error if any of these characters are included in a post. 
	//leave blank to disable this feature
	//'/([\u0400-\u04FF]|[\u0500-\u05FF])+/u'
	private $unicode_filter='/([\u0400-\u04FF]|[\u0500-\u05FF])+/u';
	
	//what percentage of the post must be non-unicode character (mosly useful for english language sites)
	//leave blank to disable this feature
	private $minimum_nonunicode_text=0.95; //percent of post that must be non-unicode
	
	//where can the user get more help about the filter? Could be a forum post or web page
	//if blank the link will NOT be added to the error messages
	//leave blank to no use this features
	private $help_url = 'http://dangerousprototypes.com/forum/viewtopic.php?f=2&t=1846&p=17886#p17886';
	
	private $show_trigger_word=true; //show the user the word that triggered the error
		
	private $load_values_from_db=false; //enable to use with DB, or use the values below
	private $minimum_days=1; //minimum days as member to post links
	private $minimum_posts=1; //minimum posts for member to post links
	
	private $sleeper_check=true; //users with 0 posts who try to post after minimum_days will be prohibited
	
	//-- Reporting variables--//
	public $filter_user=false; //we decided to filter this user (they met our criteria)
	
	public $found_stuff=false; //did we find anything? (any item below)
	public $found_sleeper=false;//did we determine this user to be a sleeper agent?
	public $found_links=false; //we found links
	public $found_words=false;//we found bad words
	public $found_unicode=false;
	
	public $error=array(); //holds text error array

	
/**
*  Test if user can have a profile yet
*  returns false if they do NOT need to be filtered
*/
function link_filter_test_profile(){
	global $user;
	
	//do we need to check this user?
	if(!$this->link_filter_check()) return false; //don't check, no error
	
	if($this->found_sleeper) return true; //if it is a sleeper agent just return error, don;t do the check
		
	//If there isn't a phpbb3 no_link message add one
	if (empty($user->lang['NO_LINK_FOR_YOU'])){
		$user->lang['NO_PROFILE_FOR_YOU']='Antispam: You can\'t have a profile yet. You need to post a few times first.';
	}
	
	$this->error[]=$user->lang['NO_PROFILE_FOR_YOU'].' '.$this->link_filter_add_help_link();
	
	return true;
}	

/**
*  Test a submitted signiture for links and words
*  Returns true if bad things detected
*/
function link_filter_test_signiture(){
	global $user;

	//do we need to check this user?
	if(!$this->link_filter_check()) return false; //don't check, no error
	
	if($this->found_sleeper) return true; //if it is a sleeper agent just return error, don;t do the check
	
	if($this->load_values_from_db) $this->link_filter_load_list_from_db(); //load list values from DB if configured	
	
	//If there isn't a phpbb3 no_link message add one
	if (empty($user->lang['NO_LINK_FOR_YOU'])){
		$user->lang['NO_LINK_FOR_YOU']='Antispam: You can\'t have off-site URLs in your sig until you post a few times. ';
	}
	//If there isn't a phpbb3 no_word message add one
	if (empty($user->lang['NO_WORD_FOR_YOU'])){
		$user->lang['NO_WORD_FOR_YOU']='Do you kiss your mom with that mouth? We don\'t want to read that! ';
	}
	
	//make a version of the post and subject
	//need the trailing space or it can hang forever in the while loop if only using a local URL
	return $this->link_filter_test(' '.trim(utf8_normalize_nfc(request_var('signature', (string) $user->data['user_sig'], true))).' ');
	
}

/**
*  Test a submitted post for links and words
*  Returns true if bad things detected
*/
function link_filter_test_post(){
	global $user;
	
	//do we need to check this user?
	if(!$this->link_filter_check()) return false; //don't check, no error
	
	if($this->found_sleeper) return true; //if it is a sleeper agent just return error, don;t do the check

	if($this->load_values_from_db) $this->link_filter_load_list_from_db(); //load list values from DB if configured	

	//If there isn't a phpbb3 no_link message add one
	if (empty($user->lang['NO_LINK_FOR_YOU'])){
		$user->lang['NO_LINK_FOR_YOU']='Your post looks too spamy for a new user, please remove off-site URLs.';
	}
	//If there isn't a phpbb3 no_word message add one
	if (empty($user->lang['NO_WORD_FOR_YOU'])){
		$user->lang['NO_WORD_FOR_YOU']='Your post looks too spamy for a new user, please remove bad words or non-english text.';
	}
	
	//make a version of the post and subject
	return $this->link_filter_test(' '.trim(utf8_normalize_nfc(request_var('message', '', true)).' '.utf8_normalize_nfc(request_var('subject', '', true))));
}

/**
* 	Do we need to check this user?
* 
*/
function link_filter_check()
{
	global $user;

	if($this->load_values_from_db){ //use MOD setting from database
		$this->minimum_days=$config['links_after_num_days'];
		$this->minimum_posts=$config['links_after_num_posts'];
	}

	//check if the user meets filter criteria
	$this->filter_user=((!$user->data['session_admin']) && (($user->data['user_type']==USER_IGNORE) || ($user->data['user_id']==ANONYMOUS) || ($user->data['user_posts']<=($this->minimum_posts-1)) || ($user->data['user_regdate']>((time())-(86400*$this->minimum_days)))));

	//this MIGHT be used in a bigger filter to only apply to new users.
	//$this->filter_user=((!$user->data['session_admin']) && (($user->data['user_type']==USER_IGNORE) || ($user->data['user_id']==ANONYMOUS) || ($user->data['user_new']==1 &&(($user->data['user_posts']<=($this->minimum_posts-1)) || ($user->data['user_regdate']>((time())-(86400*$this->minimum_days)))))));

	//check for spammers who come back later and  try to post (sleeper agents, or aged accounts)
	if(($this->sleeper_check) && ($user->data['user_posts']==0) && ($user->data['user_regdate']<=((time())-(86400*$this->minimum_days)))){
		$this->found_sleeper=$this->found_stuff=true;
		
		//If there isn't a phpbb3 sleeper agent message add one
		if (empty($user->lang['NO_SLEEPER_SPAM_FOR_YOU'])){
			$user->lang['NO_SLEEPER_SPAM_FOR_YOU']='Antispam: account disabled, please contact an admin.';
			//could delete the user automatically here
		}
		$this->error[]=$user->lang['NO_SLEEPER_SPAM_FOR_YOU'].' '.$this->link_filter_add_help_link();
	}
	
	//If you're not special, we filter you
	return $this->filter_user;
}

/**
* 	Add a help link if it exists
*  
*/
function link_filter_add_help_link(){
	if(!empty($this->help_url)){
	
		//If there isn't a phpbb3 message add one
		if (empty($user->lang['HELP_LINK'])){
			$user->lang['HELP_LINK']='Click for help';
		}
		
		return '<a href="'.$this->help_url.'">'.$user->lang['HELP_LINK'].'</a>.';
	}
}

/**
* 	if database is enabled this will explore the filter lists into the arrays
*  We move it here so this only happen when we need to filter the user
*/
function link_filter_load_list_from_db(){
	//use MOD setting from database
	$this->whitelist_urls=explode(",", $config['links_allow_always']);
	$this->no_link_strings=explode(",", $config['links_link_strings']);
	$this->no_word_strings=explode(",", $config['links_word_strings']);		
}

/**
* 	Search the text for forbidden URLs and text. 
*	Add an error to the local error array if found
*	returns true for bad stuff, false for no flags found
* 
*/
function link_filter_test($no_link_message){
	global $user, $config;

	//filter the looozers
	//remove line feeds and stuff
	$no_link_message=str_replace('\n', ' ',$no_link_message);
	$no_link_message=str_replace('\r', ' ', $no_link_message);
	//replace double spaces with single spaces (not sure why, white space?)
	while (strpos($no_link_message, '  ')){
		$no_link_message=str_replace('  ', ' ', $no_link_message);
	}
	
	//remove any own-site references, these are ok
	//first change http://mysite.com to mysite.com so we only have to look once below
	$no_link_message=str_replace($config['server_protocol'].$config['server_name'], $config['server_name'], $no_link_message);

	//whitelist other common domains too
	//we do this by relacing them with our own domain so we only have to run the search once below
	for ($x=0;$x<sizeof($this->whitelist_urls);$x++){
		if(stripos($no_link_message, $this->whitelist_urls[$x])){
			$no_link_message=str_ireplace($this->whitelist_urls[$x], $config['server_name'], $no_link_message);	
		}
	}
	
	//look at all instances of mysite.com
	while ($ok_start=stripos($no_link_message, $config['server_name'])){ //start of mysite.com
		$ok_end=strpos($no_link_message, '[', $ok_start); //find next [ (bbcode?)
		if (!$ok_end){ //if not bbcode
			$ok_end=strpos($no_link_message, ' ', $ok_start); //end is position of next space
		}
		if ($ok_end){
			$no_link_message=substr($no_link_message, 0, $ok_start).substr($no_link_message, $ok_end);//remove own URL
		}
	}
		

	//search for each link element, throw an error when found
	for ($x=0;$x<sizeof($this->no_link_strings);$x++){
		if (stripos($no_link_message, $this->no_link_strings[$x])){
			
			$this->found_links=$this->found_stuff=true;
						
			$this->error[]=$user->lang['NO_LINK_FOR_YOU'].' '.$this->link_filter_add_help_link();
			
			//$x=sizeof($no_link_strings);
			break;//no reason to go further
		}
	}
	
	//search each word, throw an error when found
	for ($x=0;$x<sizeof($this->no_word_strings);$x++){
		if (stripos($no_link_message, $this->no_word_strings[$x])){
		
			$this->found_words=$this->found_stuff=true;
		
			if($this->show_trigger_word){//show the cause so the user isn't stumped 
				$this->error[]=$user->lang['NO_WORD_FOR_YOU'].' ('.$this->no_word_strings[$x].') '.$this->link_filter_add_help_link();
			}else{//don't show the cause
				$this->error[]=$user->lang['NO_WORD_FOR_YOU'].' '.$this->link_filter_add_help_link();
			}
			
			//$x=sizeof($no_link_strings);
			break;//no reason to search further
		}
	}
	
	if($this->found_stuff==true) return true; //don't continue checking below
	
	//make a smaller subset for preg_match to save time and cycles
	if(strlen($no_link_message) > 512){
		$no_link_message = substr($no_link_message, 0, 512);
	}
	
	//Check for unicode characters that don't belong in the language of this forum	
	if(!empty($this->unicode_filter){
		if(preg_match($this->unicode_filter, $no_link_message, $m)==1){ //test for unicode character ranged defined by user
		
			$this->found_unicode=$this->found_stuff=true;

			$this->error[]=$user->lang['NO_WORD_FOR_YOU'].' '.$this->link_filter_add_help_link();
			
			return true; //don't continue checking below
		}
	}
	
	//check the percentage of the text that is NOT unicode
	//see http://www.mawhorter.net/web-development/easily-detecting-if-a-block-of-text-is-written-in-english-non-unicode-languages
	if((!empty($this->minimum_nonunicode_text))){

		//found the length in unicode mode
		$ulen = preg_match_all("#.#u", $no_link_message, $m);
		//find length without unicode
		$len  = preg_match_all('#.#', $no_link_message, $m);
		
		//determine if % of non-unicode is enough
		if(($ulen/$len)<$this->minimum_nonunicode_text){
			$this->found_unicode=$this->found_stuff=true;
			$this->error[]=$user->lang['NO_WORD_FOR_YOU'].' '.$this->link_filter_add_help_link();
		}	
	}

	return $this->found_stuff;

}

function link_filter_purge_zombies(){
	//these are from acp_prune.php functions, probably don;t need them all
	global $db, $user, $auth, $template, $cache;
	global $config, $phpbb_root_path, $phpbb_admin_path, $phpEx;

	// Get bot ids
	$sql = 'SELECT user_id
		FROM ' . BOTS_TABLE;
	$result = $db->sql_query($sql);

	$bot_ids = array();
	while ($row = $db->sql_fetchrow($result))
	{
		$bot_ids[] = $row['user_id'];
	}
	$db->sql_freeresult($result);

	// Select the group of users to delete
	$sql = 'SELECT user_id, username
		FROM ' . USERS_TABLE . '
		WHERE user_id <> ' . ANONYMOUS . '
			AND user_type <> ' . USER_FOUNDER .'
			AND user_regdate < ' . gmmktime(0, 0, 0, date("m"), date("d")-1, date("Y"))'
			AND user_posts = 0'; 
	$result = $db->sql_query($sql);

	$usernames = array();

	//this would be simpler with a SQL statment, but recycling the ppBB prune functon makes it more robust
	while ($row = $db->sql_fetchrow($result))
	{
		// Do not prune bots and the user currently pruning.
		if ($row['user_id'] != $user->data['user_id'] && !in_array($row['user_id'], $bot_ids))
		{
			//user_delete('remove', $row['user_id']);//delete the user and all posts (there should be none though)
			user_delete('retain', $row['user_id'],$row['username']); //delete users but not posts, safer just in case
			$usernames[]=$row['username'];//keep the list for the log
		}
	}
	$db->sql_freeresult($result);
	
	//add log message
	add_log('admin', 'LOG_PRUNE_USER_DEL_DEL', implode(', ', $usernames));

}

}//class

?>