Let's get serious about spam !

[Dave Bean]

I don't know about you, but I'm a little fed up with being a spam receptacle.

Steps I've put into practice:
(1) Visual Confirmation of Registation
(2) Modified agreed variables as discussed here:
http://www.phpbb.com/phpBB/viewtopic.php?p=1404100
(3) Changed membership list so members without posts are not displayed and a separate list is available for admins which show all members including those without posts

Steps (and help) Needed
(1) Better - harder to crack Visual Confirmation
(2) Confirmation option on posts as well as registration
(3) Default installation to NOT list members without posts
(4) Easy De-Spam Button desperately needed:
http://www.phpbb.com/phpBB/viewtopic.php?t=381112
- we need to be able to get rid of spam in less time than it takes to create it
(5) In my situation, where I don't know most forum registrants, we need registration information emailed to the admin with a short statement by the registrant indicating why they want to be a member for the admin activation to be efficient and effective - I just tried a week of admin activation and dropped it in favor of a review and delete approach ( sigh )
(6) Spam needs to be recognized as still being a big deal for phpbb and treated accordingly

[Mr. Sharkey]

Dave, I agree with you, but I think that the solution isn't going to come from a CVS release from phpBB.com, it's going to have to be people like us who modify our boards individually that make the difference.

My comments on the subject shouldn't be a mystery, I've commented in the forums here on them enough, but here they are again:

1) phpBB VC out-of-the box is no longer an effective deterrent to spam. I went to the FEECAP VC Mod four weeks ago and have had no, none, zero spam registrations or posts since (I have Guest posting turned on with the VC for Guests mod installed). phpBB isn't going to issue these difficult VC images in a release because of the problems associated with disabled users.

2) Espicom's 'agree' mod was a clever bit of code, but it is a short-term fix, one that loses effectiveness each and every time it gets mentioned here, and/or installed on a live board. You may be absolutely sure that the botters have taken note of it and are writing code to defeat it easily (the 'agree' text, or it's substitute are displayed in the code of the COPPA page, so there's no way to hide it completely).

3) Hiding unactivated/zero post members is a way to completely defeat the botter's intentions for registering, but until the majority of boards are doing this, the spam will continue "shotgun style" on the chance that a significant percentage of the spam registrations show up in live boards that still show new registrants in the memberlist without activation.

4) Admin approval is always going to be slow and annoying for registrants, with or without a section for any essay about why they are registering. Some people will register for the slightest reason, and never post. I get genuine registrations all the time, which from all apperances are from people who are interested in the subject of my forums, who never do post, and eventually get deleted after two months. I think that part of administering a board is the manual removal of deadwood users. If for no other reason than to make sure that the PM feature isn't being abused in some manner. (User never posted? No matter, PM still works, what if some illegal activity was being coordinated using your board's PM functions? [insert freaking out smilie here])

5) Your proposal for an admin "instant ban" button is interesting, but in the end, why bloat the database with usernames, email addys, and IP addresses that are like tissues, disposable one-time use? Provide an admin Memberlist with the "delete user and posts" button with a confirmation (Are You Sure?) before dropping the info from the database and you should be good to go. Once again, it's the admin's responsibility to keep the board cleaned up.

6) Doing the admin's duty would be less of an effort without the larger amounts of spam to deal with. Until someone comes up with an alternative to VC, a better, more difficult VC image, combined with unique mods to the regular phpBB releases is going to be the best we can do.

7) Severe punishments for spamming. This could be the subject of an entirely new category of forums here: "Crime and punishments, post your favorite spammer torture techniques here". Maybe having paid bounty hunters to bring spammers to justice?

8) Get used to it. The evidence provided by email spam is that it isn't going to stop anytime soon. Maybe someone will get clever and write a "Spamassassin" program mod for phpBB ??

[romans1423]

Subjecting all guest posts to Akismet.com's spam tool would seem viable enough, if someone wrote the MOD to interact with their API, which would require an admin panel listing of all blocked messages to check for and save false positives as well as a means to send back to Akismet false negatives that make it through to the board.

Something like this for registrations would be very useful.

[Rudy64]

Dave, I agree also. Good points.

As I see it, the images for the visual confirmation text are hard-coded into the PHP file, so they are easily predictable by the 'bots. I don't think it would take much effort to replace this with something similar to what other applications are using for VC. In other words, using GD or ImageMagick to generate the distorted text randomly. If I had the time to tackle it for my own board, I would. Although the mod mentioned earlier in this post might be a good way to go about it. Since I code in PHP, adding mods is no big deal for me. And if it got rid of the spambots ripping through my forums' VC and getting registered, it would be well worth it.

The only hangup I see is that, as vanilla as a phpBB install is, the phpBB folks can't assume that either (or both) GD and ImageMagick are installed on everyone's server. It is possible to detect it (as most apps I install can do that), but it would add another hurdle to the installation process for those "newbie" phpBB admins who can barely navigate a command line.

It just made me frustrated to see a spam post on our forum today, courtesy of a bot, and finding about a dozen other spambots that registered in the past couple of days...and I've had VC enabled since early March.

[Dave Bean]

Spam seems to be getting worse on a daily basis. Even with visual confirmation and and variable changes, I had 4 spam posts today and many new registrations without posts. Since I don't show members without posts on the member list, it does not help the spammers, but they are still coming in.

I'm still big on the need for the Easy De-Spam button:
http://www.phpbb.com/phpBB/viewtopic.php?t=381112
what would it take to hire someone to to this?

I think I'll move to email activation and then maybe email banning. Admin activation just made more work for me without helping the process when we do not have new member info in the email.

Perhaps the next step is no activation without the new member personally contacting me - porn spam has to stop or be deleted quickly, or we'll have to stop the forum.

.... just a thought - is there such a thing as admin post activation for members with 0 or few posts?

I am running 2.0.19 rather than 2.0.20 - are there any new anti-spam features with 2.0.20?

[drathbun]

I think I'll move to email activation

So I take it that you currently have no activation on at all?

Having email activation turned on does stop a number of the spammers. I don't even use visual confirmation, I use a combination of email activation (which some bots are still able to do, btw) and code that hides the new members from the member list + newest member, plus code that prevents members from entering a website as part of their profile until they've reached a set number of posts, plus a few other things. I also have code set in a cron job that removes inactive users after 30 days.

The bottom line is that no solution is perfect, because they can always be defeated by a human. That's sort of the point... you want humans to be able to register. I've read where some folks are having humans register (and humans obviously should have no problem with visual confirmation) and then the user accounts / password information is dropped into a posting database, and the posting bot takes over after that.

I'm not saying VC has not been broken, I believe I've seen more than enough to believe that it has. But it's just one step of many. The challenge, as I see it, is to make things difficult to have spammers to a mass-flood of registrations or posts, while not locking out regular users. I have good feedback from my community that if they had to enter a "posting code" each and every time they posted they would be, um, less than happy. So my moderators have to delete a spam post every now and then.

Just some food for thought.

[Rudy64]

Before I go on...has anyone ever thought of renaming the profile.php file to some random name, and modifying the appropriate other files in phpBB to reflect this? Even the "mode=register" could be munged a bit.

Anyway...

I'm not saying VC has not been broken, I believe I've seen more than enough to believe that it has. But it's just one step of many. The challenge, as I see it, is to make things difficult to have spammers to a mass-flood of registrations or posts, while not locking out regular users.

You've got it--I'm changing to a different VC system, but that does not mean I'm going to overlook the other tips I've been reading this morning about securing phpBB further. There are plenty of threads here I am looking at to help cut down on the spamming, and try to prevent it on forums the spambots have not discovered yet.

I've always had the feeling that if you make something difficult enough, most will find an easier target. And on the flip side: anyone who is persistent enough IS going to get around the system, no matter what we do.

I made the mistake on a newer forum I set up recently: I forgot to set the account confirmation to "e-mail". Yeeeesh, did we have the spam!! Fortunately we caught it early on. It is interesting to note that the old forum was run on WebBBS, and while it was a completely different system, it seems that we were on some kind of forum listing and the spammers found the new forum within a day or so.

It's disgusting to think of all the (billable) hours I've wasted having to deal with e-mail filtering, altering scripts to prevent spam, modifying phpBB, etc. just to keep this stuff down. Incredible waste of time, bandwidth and money. It's sick.

[Dave Bean]

Hi drathbun - thanks for the reply.

Part of getting serious is sharing information. I suppose, that I should experiment with user email activation - why not?

Situations will vary for different forums, however in my case, the more I think about it, the more I think that 2 things will solve the problem.

(1) Don't display members without posts or some other new member criteria (I do this) and not show latest member (don't know how to do this)

(2) Have an option for admin activation of post for new member or member of a certain rank.

#2 above might not only solve the problem for new member spammers, but if necessary we might beable to review selected old members that need some filtering.

For admin (or moderator) activation of new member posts, would we need a custom field in the post or perhaps the member tables? ... or maybe it would be easier to save pending posts in a pending table.

No matter what we do, some objectionable posts will make it through. We could make it easy on ourselves by segmenting members, identifying which member posts need to be reviewed before posting and which posts belong to members that have gained our trust.

I realize the post activation is extreme and all boards will not wish to do this, but for boards where posting spam is a problem in addition to registration spam - this should minimize the problem and effort.

In our case we syndicate topics to another website so porn spam reflects very poorly on us and we just can't have it.

[drathbun]

(1) ... and not show latest member (don't know how to do this)

Alter includes/functions.php as follows:
Change

Code: Select all

		case 'newestuser':
			$sql = "SELECT user_id, username
				FROM " . USERS_TABLE . "
				WHERE user_id <> " . ANONYMOUS . "
				ORDER BY user_id DESC
				LIMIT 1";

To

Code: Select all

		case 'newestuser':
			$sql = "SELECT user_id, username
				FROM " . USERS_TABLE . "
				WHERE user_id <> " . ANONYMOUS . "
				AND user_active = 1
				ORDER BY user_id DESC
				LIMIT 1";

In our case we syndicate topics to another website so porn spam reflects very poorly on us and we just can't have it.

In that case, you must be running some sort of RSS or other process, which is external to phpBB, yes? Perhaps you can provide an "approval" process there rather than on phpBB. With more details I can perhaps try to help.

[NeoThermic]

phpBB isn't going to issue these difficult VC images in a release because of the problems associated with disabled users.

I'm not overly sure where you got that idea from. The CAPTCHA for 3.0 is not the same as the 2.0 one, and I would wager that after extensive testing of 3.0's CATPCHA that it would be a candidate for backporting into 2.0.x

I went to the FEECAP VC Mod four weeks ago and have had no, none, zero spam registrations or posts since (I have Guest posting turned on with the VC for Guests mod installed).

I'll also note that freecap itself isn't immune to automated cracking, as per my topic over at area51:
http://area51.phpbb.com/phpBB/viewtopic ... 03#p141803

NeoThermic

[Dave Bean]

drathbun - thanks for the newest user exclude inactive - closes that gap

Spammers members with no posts
-----------------------
I think with the membership list only showing members with posts (or active for some forums) and the newest user exclusion to having posts (or active). We've eliminated benefits for spammers to register, but not post and we can just clear them out once a month.

Spammers with posts
---------------------------
These are almost exclusively new members. So we need some kind of admin or moderator approval of posts made by members that have not gained our trust yet. I found the Freeze Posts mod which is in the direction of what is needed for spam:

If Freeze Status is Enabled in a Forum, all posts made by a normal user on that forum must be validate from a
moderator before to become visible for all users.

http://www.phpbb.com/phpBB/viewtopic.php?t=381964
Perhaps we could elevate trusted members to a level not requiring validation for posts?? Comments - wouldn't this put a big dent in the spam posts while letting trusted users continue the exchange?

[Mr. Sharkey]

phpBB isn't going to issue these difficult VC images in a release because of the problems associated with disabled users.

I'm not overly sure where you got that idea from.

Well, it's an opinion I formed after installing the freecap mod. Even after viewing literally dozens of the VC images during installation and testing, I still have difficulty reading the image and can only successfully enter it in the required field about 60% of the time. I can't see phpBB releasing a version of VC that's that difficult.

I'll also note that freecap itself isn't immune to automated cracking, as per my topic over at area51

Never expected it to be 100% foolproof, but compared to the out-of-the-box phpBB VC images, it seems to be doing the job (for now, I'm sure it'll get compromised eventually).

[the_host]

thanks for the heads up on this, im going to implement some of the things you guys have suggested here on some of my sites.

[Dave Bean]

If deterring forum spam was easy, we wouldn't be having this discussion. The initial topic was "Let's get serious about spam". In my opinion the forum spam problem is huge, to the extent that as it continues to increase in volume and sophistication, spam will be fatal to boards like I run, which are open to the general public.

The solution is 2 steps:
(1) Totally eliminate the effect of the members without posts spam by not listing members without posts in the member list or in the newest member display. This approach should be the default and should put an end to the member list spam problem. (we have this solution available as a mod now and it needs to become a standard default option in phpBB)

(2) The spam battle will be in the area of posts with #1 above in place. We certainly will do all the image verification, activation hurdles, required registration to post, moderator and admin post reviews which will keep a lot of spam out. However, in the end, as with email spam, the spammers will post. As in spam email, forum spam on my boards now exceed non spam posts.

I think a popular email anti-spam approach may work with forum spam. We have white list posters (have gained our trust), black list posters that we ban now, and we need a grey list poster designation. There needs to be an option for Posts by grey list members who have not yet gained our trust to be placed in a pending file, hidden from the public, that will be reviewed by moderators and admins. The admin or moderator would then either allow the message to be made public or delete the member and all of their pending posts (hopefully with one click) if they are not appropriate for the board. (I'm not quite sure how to implement this - there is a mod to hold Guest posts, which may be similar, but looks complicated to inact - a feature such as the grey list needs to be standard phpBB)

Seems like the benefits from the above 2 anti-spam tools will greatly exceed the effort that will be required to implement the tools.

Comments?

p.s. I cleanned out spam before entering the above post. In the time that it took to post the above message, 2 more spammers defeated the image verification, email activation and posted spam. My notify on new post picked it up and I deleted the 2 spam posts and users - really need #2 above.

[brainsys]

I would also like to add, (for the nth time), that 'rel="nofollow"' be included by default in all links (admin overidable).

The only way to combat spamlinks over time is to make them useless. The financial incentive to get Googlerankings will outwit anything the average sysadmin can do to stop them. as long as the majority of phpbb boards are vulnerable - we will all be under an increasingly sophisticated attack.

I do hope the phpbb strategists are talking with Google to neuter the threat to our mutual advantage.

Sorry to criticise but all I see here is people fighting battles - not planning how to win the war. Perhaps I'm just a bit grumpy because I had a couple of spammers attacking my board during the Champions League final last night.

Oh - and congratulations Barcelona, you only just beat 10 men

Stuart