[01/12/06] Spam Words

Joe Belmaati · Post by **Joe Belmaati** » Fri Oct 05, 2007 12:55 pm

Hi,
sorry about the silence. Unfortunately I will not be able to rewrite the MOD for phpBB 3.0 - There simply aren't enough hours in the day for me to get familiar with phpbBB3 core code. As for the log, lancevance probably means that the log page never gets around to loading, therefore he can't use the empty log feature. A simple solution would be to simply run this query in phpMyAdmin

TRUNCATE phpbb_spam_words_log;

That'll empty out the log and reset the counter.

As for maximum spamword entries, then currently the restriction is at 10,000 entries but that can be changed by running a simple SQL query. If that ever becomes an issue then let me know.

bruster42 · Post by **bruster42** » Mon Nov 12, 2007 1:31 am

Please help. I've created the tables, made all the SQL changes, now I'm starting on the file mods.

The first one in "posting.php" says FIND
if ( $error_msg == '' )
{

Problem is there are a number of these in this file. Which one do I modify? There's no "about line #" reference.

Thanks.
Bruce

will_hough · Post by **will_hough** » Mon Nov 12, 2007 3:24 am

bruster42 wrote:Please help. I've created the tables, made all the SQL changes, now I'm starting on the file mods.

The first one in "posting.php" says FIND
if ( $error_msg == '' )
{

Problem is there are a number of these in this file. Which one do I modify? There's no "about line #" reference.

Thanks.
Bruce

Line 567 in a vanilla posting.php file.
There is only one instance of

Code: Select all

			if ( $error_msg == '' )
			{

in posting.php If you have more than one, your file may not work.

JohnSin · Post by **JohnSin** » Wed Dec 05, 2007 4:39 am

Hi. Love this mod!

However it doesn't stop flagged posts from showing up in RSS feeds. Any idea how I might extend it to do that?

Thanks in advance!

John
http://www.seedcode.com

Joe Belmaati · Post by **Joe Belmaati** » Wed Dec 05, 2007 5:48 pm

Hi,
sure, that's possible. I think I posted a hack for one of the RSS mods a while back. Look back through the pages and you should find it. Anyhow it's just a matter of adding a clause to the query that fetches posts for the feed.

JohnSin · Post by **JohnSin** » Wed Dec 05, 2007 6:27 pm

Thanks for the quick reply. I searched for your posts and "rss" and couldn't find anything that would show me the clause to add (still pretty new to this). But found a post that suggested showing an error message instead of flagging (as with the error message the post is never written out to rss) so I'll try that. Thanks again.

John

Nothing is easy · Post by **Nothing is easy** » Thu Dec 06, 2007 3:29 am

Hi Jo.
Quite a while ago I installed your patch to check registration fields for spam which works great. Later I suggested a later release enhancement to write a log entry for those that get caught at registration time. Anyway I decided to hack that feature in a couple of days ago and have that working ok.

Being able to see what gets caught at registration has certainly put my mind at rest because I have around 1500 spam words set up and there was always the fear that I was stopping good registrations without knowing.

However, I have 1 question.

For the registration field check you use array_map('spamcheck', $check_fields) which calls the function for each of the parameters in the array. The impact of that is the entire spam words table is read several times (once for each of the parameters). Also logging what all of the registration fields were is a lot harder because only the one parameter is available at the time spam is detected.

So my question is was there a reason why you decided not to concatenate all the fields together separated by a space and pass them as one field into the check? If done that way then the database is only read once and all fields are available to write into a log entry all be in in a concatenated string.

It's a genuine question Jo and not meant to be critical. I'm not a trained programmer, just self taught, and was wondering before I tried that if there was a potential problem.

Joe Belmaati · Post by **Joe Belmaati** » Thu Dec 06, 2007 7:14 pm

Can't remember the exact code segment, but normally I would never write code that runs more than one query. Does the function spamcheck() run a query? Logic would be that the spamwords table is read once and the result shoved into an array. The idea is then to read the array, not run another query. Anyhow, I think what spamcheck() does is to read the array for every field that is checked. That would make sense. BTW, I am also a self taught programmer. My real profession is being a musician

John, I can easily modify your rss file. Post a link to the MOD and I will write it up for you. This question seems to creep up from time to time, so it makes sense to have it write here in the thread.

JohnSin · Post by **JohnSin** » Thu Dec 06, 2007 7:22 pm

Thanks Joe: very cool of you. We're using a slightly older version of this:

http://www.phpbb.com/mods/db/index.php? ... ib_id=1253

Thanks again!

Joe Belmaati · Post by **Joe Belmaati** » Thu Dec 06, 2007 7:39 pm

OPEN rss.php

FIND

Code: Select all

                ORDER BY t.topic_last_post_id DESC

BEFORE, ADD

Code: Select all

                AND p.post_flagged <> " . TRUE . "

JohnSin · Post by **JohnSin** » Thu Dec 06, 2007 7:50 pm

LOL! Thanks!

Nothing is easy · Post by **Nothing is easy** » Sun Dec 09, 2007 3:15 pm

Joe Belmaati wrote:Can't remember the exact code segment, but normally I would never write code that runs more than one query. Does the function spamcheck() run a query? Logic would be that the spamwords table is read once and the result shoved into an array. The idea is then to read the array, not run another query. Anyhow, I think what spamcheck() does is to read the array for every field that is checked. That would make sense. BTW, I am also a self taught programmer. My real profession is being a musician

Well all I can say Joe is that your mod has been so fantastic that it has been sweet music to my ears

The code at registration check in usercp_register.php was:

Code: Select all

function spamcheck($param) 
{ 
   global $db; 
    
   if(@strpos(strtolower($param), '[url') !== false) 
   { 
      message_die(GENERAL_ERROR, 'No url allowed'); 
   } 

   $sql = "SELECT * FROM " . SPAM_WORDS_TABLE; 
   if (!($result = $db->sql_query($sql))) 
   { 
      message_die(GENERAL_ERROR, 'Could not get spam words from database', '', __LINE__, __FILE__, $sql); 
   } 

   $param = preg_replace("#\[.{12,16}\]#i", '', $param); 
   $param = preg_replace('/\[url\]|\[\/url\]/si', '', $param); 
   $param = preg_replace('#\[url=|\]|\[/url\]#si', '', $param); 
    
   if ($row = $db->sql_fetchrow($result)) 
   { 
      do 
      { 
         if (preg_match('#\b(' . str_replace('\*', '\w*?', phpbb_preg_quote($row['spam_word'], '#')) . ')\b#i', $param)) 
         { 
            message_die(GENERAL_ERROR, 'Not allowed'); 
         } 
      } 
      while ($row = $db->sql_fetchrow($result)); 
   } 
    
}

and the call to the function was:

Code: Select all

if($mode == 'register') 
{ 
   $check_fields = array($username, $email, $icq, $aim, $msn, $yim, $website, $location, $occupation, $interests, $signature); 
   array_map('spamcheck', $check_fields); 
}

So the way I read that is that the function is called once for each array entry and will cause the table to be read each time and the only parameter available to the function is the array entry being evaluated.

Joe Belmaati · Post by **Joe Belmaati** » Sun Dec 09, 2007 6:10 pm

That is correct. The function is called once for each array element, so 11 times I guess. So, essentially you're right, 11 queries...

I'm sure it could be rewritten to something more efficient. Maybe it was just a quick fix that I posted.

homer09001 · Post by **homer09001** » Mon Dec 17, 2007 12:09 am

found a bug: if a word in the spam list has a ! or a ? in the filter doesnt pick the word in the post?

homer09001 · Post by **homer09001** » Mon Dec 17, 2007 12:36 am

Complete list of 674 Spam Words
i have compiled my list of Spam Words into one file for all, it was created using words i researched on the net, words i added myself and Millennyum's complete list. (Thanks Millennyum

)

http://rs1.rapidshare.com/files/7708770 ... _words.sql

[edit]updated download to new sql file with fixed errors

Rating:

advertisements