[01/12/06] Spam Words

All new MODs released in our MOD Database will be announced in here. All support for released MODs needs to take place in here. No new MODs will be accepted into the MOD Database for phpBB2
Forum rules
READ: phpBB.com Board-Wide Rules and Regulations

On February 1, 2009 this forum will be set to read only as part of retiring of phpBB2.

Rating:

Excellent!
52
76%
Very Good
9
13%
Good
4
6%
Fair
0
No votes
Poor
3
4%
 
Total votes: 68

i-world
Registered User
Posts: 7
Joined: Fri Aug 11, 2006 7:13 am

Post by i-world » Sat Sep 23, 2006 4:14 am

erinspice wrote: I just wanted to voice my support for an update. I'd love to get this mod working on my forum, but I simply don't have time to sort through 16 pages of posts after the initial install searching for more code changes. I have the mod without any of the changes posted to this thread and in 3 months it has not stopped a single spam from getting through. I get 3-5 spam per day on my forum, and I'm really looking forward to that update!


Something is wrong with your installation, I think. When I installed this without any changes from the 16 pages of discussion and code it still stopped most spam for me. That wasn't reflected on the index listing, but they were flagged just fine.

You should probably go back through the install directions and check for the code changes, then check you list of words. Mine isn't even close to some people on here in length and still catches most spam. When it doesn't it's usually because I need a new word/pattern in the list.

The one thing I think I am ready to track down from the forum here is code that flags anything with too many links. I'm starting to see more of that and it's not always caught by listed words.

User avatar
bacon tacon
Registered User
Posts: 236
Joined: Sun Oct 27, 2002 8:17 pm
Contact:

Post by bacon tacon » Sat Sep 23, 2006 10:17 pm

Some Spam just got through even though I'm blocking multiple links. I copied and tried to post it myself, but the link blocking worked. (The Spam related to online gaming and my site is about vintage slot machines so I find these hard to word filter).

Recently some sex Spam got through but was removed by a moderator before I got a peek. This must have bypassed both link and spam word blocking.

Maybe Joe's theory's correct - they're somehow posting auto converted html entities that don't match any of the patterns we're looking for.

saaiberke
Registered User
Posts: 1126
Joined: Wed Jul 20, 2005 8:13 am
Location: Gent/Belgium
Contact:

Post by saaiberke » Sun Sep 24, 2006 11:51 am

Excellent Mod! Thank you Joe!

I have one little question, the PM function to announce members that they used a spamlisted word, is there any possibility to have it sent by another admin? Now it's always my name that apppears, I'd rather like the name of the admin/mod team to appear.

Grtz,

Nic
Last edited by saaiberke on Mon Sep 25, 2006 8:27 pm, edited 1 time in total.

helmut
Registered User
Posts: 35
Joined: Mon Nov 10, 2003 9:18 pm
Location: Germany
Contact:

Post by helmut » Sun Sep 24, 2006 6:32 pm

erinspice wrote: I just wanted to voice my support for an update. I'd love to get this mod working on my forum, but I simply don't have time to sort through 16 pages of posts after the initial install searching for more code changes.

Pretty much what I think. Joe does as much as he can, already, so let's be patient.
erinspice wrote: I have the mod without any of the changes posted to this thread and in 3 months it has not stopped a single spam from getting through. I get 3-5 spam per day on my forum, and I'm really looking forward to that update!

Installing the mod is one initial task you have when using the spam word filter, feeding the word list is a second and a more or less permanent one with some peek at the beginning. The spam word filter comes without any spamwords at all (which is good I think). This means that during the first days/weeks you have to feed it with lots of words you want to filter out. Less and less spam will get through and after a while the permanent task of feeding the spam word filter becomes less.

Even without the code changes the spam word filter will filter out quite a lot. There will be flaws and holes, but it does filter out quite a percentage of spam. If you want to improve the spam word filter's recognition, it's sufficient to change three lines of codes in spamwords.php (see Joe's post on So Sep 17, 2006 6:54 am).

The current numbers of the board I administer:
~500 entries in spam word list
~50-60 spam posts per day.
~90% flagged as spam (a good value, I think)
- Guest posting is allowed (on purpose)

My impression is that depending on the actuality of the spam word list, the activity of the forum, and the focus of the spam bots, these numbers vary a lot.
Last edited by helmut on Mon Oct 02, 2006 7:37 am, edited 2 times in total.

i-world
Registered User
Posts: 7
Joined: Fri Aug 11, 2006 7:13 am

Post by i-world » Mon Sep 25, 2006 7:49 pm

There has been a lot of code thrown around for either spam words in links or too many links in a post.

All that was too confusing for me but I found another mod for that problem which works well for me. There is one that prevents guests from posting any links at all.

My problems are probably 80% guest posts so that took care of that problem. I've pretty much eliminated automatic bot registrations using a couple other mods so I have little problem with them registering then posting.

So this mod is doing a great job with its original intent, and a few other problems are solved well with other mods. I found this easier than trying to sort through too much code here. With the exception of the fix for not indicating new posts if they are flagged - that change is key to the function of this mod.

Just thought this might help a few people looking for solutions to other problems by recoding this mod. If you have a different problem, even a related one, check the mods db and you might find something helpful. It seems like everytime a new problem becomes more common a new mod to deal with shows up before long.

And thanks again for this one Joe, I'm finding it does what its supposed to very well and it makes a huge difference.

Joe Belmaati
Registered User
Posts: 2110
Joined: Sun Sep 28, 2003 7:35 pm
Location: Denmark

Post by Joe Belmaati » Mon Sep 25, 2006 8:05 pm

Thank you very much. I submitted an update to the MODs database yesterday.
##
## 2006-09-24 - 1.1.0
## - bug fixes
## - added: mass ad words
## - added: mass delete flagged posts
## - added: ability to choose pm sender from list of admins
##


All the code that has been thrown around here has been applied judiciously and all the "additional changes" such as not showing the last flagged post on index has got their own install file which can be deployed on a need basis. Here's to hoping the new version will pass validation. We'll know in some weeks.
Image

windowsceportal
Registered User
Posts: 10
Joined: Thu Mar 17, 2005 7:00 pm
Location: Hungary
Contact:

I've installed Spam Words 1.0.2 into my phpBB 2.0.18

Post by windowsceportal » Mon Sep 25, 2006 8:59 pm

I've installed Spam Words 1.0.2 into my phpBB 2.0.18. It doesn't work properly. After that, I checked the installation steps (INSTALL.mod) several times, but I performed all of steps as it wrotes. Spam Words menu appeared in admin area. So, this is OK.

The forum is working but if i try to post a spam post (for testing porpuse) it doesn't send me any kind of error message and saves the post. The spam words database is not empty and the spam error message is enabled in settings panel.

Any idea how to solve this problem?
Windows CE Portál: http://www.windowsceportal.hu

clawed
Registered User
Posts: 32
Joined: Tue Jun 14, 2005 3:04 am

Post by clawed » Tue Sep 26, 2006 12:11 am

I have been having a problem with a few spam posts and have installed this mod and it works well. It would be nice to add spam words in a text area one per line as it takes a while adding them one by one.

On a side note I would like to create a mod like this but use a bayesian filter (PHP bayesian filter) and have 2 bottons on every post that only moderators only see "spam" and "not spam" and automatically move spam posts to a certain section of the forum (that only moderators can see). Has anyone tried anything like this? does anyone want to help me?

[Tee]derDoc
Registered User
Posts: 11
Joined: Wed Sep 04, 2002 5:15 pm
Location: somewhere in the cold nothern part of Germany

Post by [Tee]derDoc » Tue Sep 26, 2006 4:33 am

@clawed: I'd love to help... that sounds like one of the best ideas to me, my Mozilla's been catching close to 100% of all the spam I still get, even after 2 levels of filtering ;)

@nampa regarding highlighting of word in spam-post:
One way around this is to highlight the word already when entering it into the log.
I dont remember if the message only gets entered into the log, if it isnt flagged, or if it is entered anyway...
If its always entered regardless of being flagged or not, just highlight the word while in the mathing loop, that gets rid of the pain of doing that later while displaying, that is what I did in this code here:

Code: Select all

if(	($i=preg_match_all($word, $tot_mess, $m)) ){
				$off_words++;
				$times_words .= (($times_words)?',':'') . $i;
				//replace the word in the message by a highlighted one
				$sub = preg_replace($word, "${bef}$0${aft}", $sub);
				$mess = preg_replace($word, "${bef}$0${aft}", $mess);
		
			}
You will notice a couple of things:
I count the number of times the word actually matched for later displaying;
$tot_mess is the total msg, made by concatenating sub,msg and sig;
and after matching the subject and message get the word replaced, surrounded by $bef and $aft, which is a span that sets the font bold and red.

Another thing that I did, is to get rid of all html special chars before matching, that way spammers cant get away with postings html-entities that dont get matched.
This can simply be done by these 3 lines:

Code: Select all

$message = html_entity_decode($message);
$subject = html_entity_decode($subject);
$signature = html_entity_decode($signature);
A TRUE Klingon Warrior does
not comment his code!

Joe Belmaati
Registered User
Posts: 2110
Joined: Sun Sep 28, 2003 7:35 pm
Location: Denmark

Post by Joe Belmaati » Tue Sep 26, 2006 5:58 am

Some good ideas there. The update I submitted a few days ago contains a fix for the non-highlighted spam word in the log. It seems to work well.
Image

uniterre
Registered User
Posts: 29
Joined: Mon May 29, 2006 1:28 pm
Contact:

Post by uniterre » Tue Sep 26, 2006 5:10 pm

[quote="
Another thing that I did, is to get rid of all html special chars before matching, that way spammers cant get away with postings html-entities that dont get matched.
This can simply be done by these 3 lines:

Code: Select all

$message = html_entity_decode($message);
$subject = html_entity_decode($subject);
$signature = html_entity_decode($signature);
[/quote]

Please, can you explain where we have to make this change ? in which php file ?
Maybe it could solve the problem of auto converted html entities that don't match any of the patterns we're looking for ?

Thanks,
Alexandre

[Tee]derDoc
Registered User
Posts: 11
Joined: Wed Sep 04, 2002 5:15 pm
Location: somewhere in the cold nothern part of Germany

Post by [Tee]derDoc » Thu Sep 28, 2006 5:12 am

@uniterre

Just insert them at the beginning of the function check_spam($message, ...)
in includes/spamwords.php.
Should be around line 250.. I dont know exactly, since I changed a lot in my version.
Just insert those as the first 3 lines in that function, that should do the trick ;)

-V
A TRUE Klingon Warrior does
not comment his code!

erinspice
Registered User
Posts: 14
Joined: Sat Feb 19, 2005 10:41 pm

Post by erinspice » Fri Sep 29, 2006 2:35 am

Joe Belmaati wrote: Thank you very much. I submitted an update to the MODs database yesterday.


Wonderful! Now where do I find the new package? I just used the MODS database to download a new copy of the archive file, and it has the same size and md5sum as the one I downloaded on 28 Jan 2006.

Joe Belmaati
Registered User
Posts: 2110
Joined: Sun Sep 28, 2003 7:35 pm
Location: Denmark

Post by Joe Belmaati » Fri Sep 29, 2006 3:46 am

erinspice wrote:
Joe Belmaati wrote:Thank you very much. I submitted an update to the MODs database yesterday.


Wonderful! Now where do I find the new package? I just used the MODS database to download a new copy of the archive file, and it has the same size and md5sum as the one I downloaded on 28 Jan 2006.
Once the MOD team has validated it it will be avialable for download.
Image

helmut
Registered User
Posts: 35
Joined: Mon Nov 10, 2003 9:18 pm
Location: Germany
Contact:

Post by helmut » Sat Sep 30, 2006 8:59 am

Joe Belmaati wrote: Once the MOD team has validated it it will be avialable for download.

Very good. Thank you very much for your effort, Joe! :-)

I'd even would like to download the update before evaluation is accomplished and would like to know which features have been included, if possible. Anyway, we'll wait patiently. ;-)

Post Reply

Return to “[2.0.x] MOD Database Releases”