Very curious Http Referers going to my site and being listed

The 2.0.x discussion forum has been locked; this will remain read-only. The 3.0.x discussion forum has been renamed phpBB Discussion.
Azmal
Registered User
Posts: 11
Joined: Mon Sep 11, 2006 12:59 am

Very curious Http Referers going to my site and being listed

Post by Azmal »

My forum logs and posts the top http referers and for a long while now I've been noticing strange places that the links are coming from. My forum isn't very advertised and these links aren't even from english sites so I knew there was something fishy.

I had to go out of state for a couple months and now I'm seeing those links up in the top 10 and I know they aren't just the random browser. Not only that but there are a lot of porn links in there now too....

The only thing I can think of is that maybe they're intentionally spamming sites with hits that record the referers' links so they can get a higher ranking in search engine results. There have also been frequent users posting porn links lately and I'm willing to bet it isn't bots doing it.

Removed as there are links to things young eyes should not see - or mine>Jim_UK. :wink:
Here is the direct link to my site to see what I'm talking about.

Has anyone else experienced anything like this? I want to keep everything intact but get rid of that nonsense and block those sites somehow. Should I seek professional help, if so, where would I begin looking for that?
Thanks in advance

alvo
Registered User
Posts: 713
Joined: Thu Jun 22, 2006 3:57 am

Post by alvo »

All web servers log referrers or every request made to them. Few people make it publicly available as there's little reason to do so and the bot scripts aren't going to know if you do so or not, in fact the bot script authors probably don't know. All that extra hits gets them is a higher number after their URL, but still only one mention of the URL.

The best plan of attack is to stop the bots from being able to register/post and hide the URLs in the member list from those that never posted. There's no easy way to match up referrers with those that posted, so they'll remain in that list as long as you make it available.

RoXaZer
Registered User
Posts: 1
Joined: Sun Dec 03, 2006 6:23 pm

Post by RoXaZer »

I have exactly the same problem and i will turn off so users dont can register at the forum, but i dont know how? O_o

Azmal
Registered User
Posts: 11
Joined: Mon Sep 11, 2006 12:59 am

Post by Azmal »

Well there's that; the ones who come in and add their porn site in their profile, and often post one message which makes them active. I delete that mess as best I can.

What I'm talking about here is different and I'm really very curious as to why, or even how, there are so many hits racked up from these sites. Non english sites at that, when all of the users on my forum are english speaking.

The link I posted was removed but there are thousands and thousands of hits from these porn sites and what seems like to be spam advertising websites (I assume because they have /camera and /casino at the end of the xoomer.alice.it/casino for example or whatever their domains are.

I really have no clue about whats happening or why I would get so many hits from these places. Ultimately I may just have to strip the sections that list the incoming urls and hits but I actually like it up when its not filled with crud. Even then, it will still be polluting the website's stats when I check the server stats. How in the world can I fix this? Is it even possible to actually block hits coming from a certain URL? even then it'd still be a pain because there's so many.... Its so strange, there has to be SOME motive.... Drives me nuts... Anyone have any clue about this?

romans1423
Registered User
Posts: 1552
Joined: Sat Nov 02, 2002 4:44 pm
Location: Connersville, IN
Name: Rick Beckman
Contact:

Post by romans1423 »

Sounds like you got a nasty case of referrer spam.

Using Bad Behavior helps.

Simply place the Bad-Behavior directory within the same folder which holds /admin/, /includes/, and /templates/ for your board. Then include the following line after the opening <?php of your board's config.php:

Code: Select all

include_once('Bad-Behavior/bad-behavior-generic.php');
Should help prevent not only referrer spam, but also most other spam you could get on your board caused by bots. (Most, not all spam, but it does help!)

alvo
Registered User
Posts: 713
Joined: Thu Jun 22, 2006 3:57 am

Post by alvo »

There's also a mod that prohibits having a link in their profile until a predetermined number of posts has been made (default is 10 but can be set to whatever you like).

romans1423
Registered User
Posts: 1552
Joined: Sat Nov 02, 2002 4:44 pm
Location: Connersville, IN
Name: Rick Beckman
Contact:

Post by romans1423 »

alvo wrote: There's also a mod that prohibits having a link in their profile until a predetermined number of posts has been made (default is 10 but can be set to whatever you like).


That isn't his problem. He's publicly displaying a list of sites which refer hits to his site, similar to the first block (upper-left) here.

But his list of referrers is being filled up not by legitimate referring sites, but by spam sites. These aren't genuine referrers but show up in the list because automated scripts with spoofed referrers are hitting his board to make it seem as though a link is on the spam site.

What this does is get the spam site listed in the HTTP logs, which some webmasters monitor, but if the list is displayed publicly (as is here the case), it gets them a free link back to their site.

This isn't profile spam; it's referrer spam. Big difference.

Azmal
Registered User
Posts: 11
Joined: Mon Sep 11, 2006 12:59 am

Post by Azmal »

Exactly! Well said, I couldnt quite put it into words.
Praise to you kind sir. Much appreiciation. I'll download that and try it asap!
Thanks again!

Azmal
Registered User
Posts: 11
Joined: Mon Sep 11, 2006 12:59 am

Post by Azmal »

Its all set up, guess the only way to test it is to wait and see. I'll post the results later on.

eccerr0r
Registered User
Posts: 20
Joined: Tue Nov 28, 2006 4:51 pm
Location: Colorado, USA
Contact:

Post by eccerr0r »

One thing is to make sure your website never displays the referers, most of the spammers are hoping that someone has a script that happily shows all the referers.

While it's interesting to look at the referers I try to keep mine hidden from all users and unavailable from the general public. People also muck with user agents the same way. There's not much that can be done, there always will be people feeding in garbage referers, but making it less worthwhile by not making this data available makes it less valueable to the spammers (since it will no longer give a google link or whatever link to them!)

I actually still have my user agents webpage up. However, I sanitize all entries being displayed, so anyone trying to insert a link gets garbage HTML that won't be parsed as a link, thus giving no value to their spam webpage.

Seems like a common theme here now: do not make it easy to store data on it that's easily retrievable by a computer in any way, shape, or form. It will be abused by spammers, warez kiddies, etc.

romans1423
Registered User
Posts: 1552
Joined: Sat Nov 02, 2002 4:44 pm
Location: Connersville, IN
Name: Rick Beckman
Contact:

Post by romans1423 »

There should be nothing wrong with displaying our referrers though; the actions of malicious users -- crackers, spammers, etc. -- ideally should not affect what we can and cannot do with our websites (or what our users can and cannot do while on them).

Displaying the referrer list publicly gives a nod to those sites which are sending traffic your way. The automatic link back to them is a great way to thank them! Further, a referrer list is a great for allowing users to find other content which they might enjoy because it is likely that sites which are sending traffic to your site are in some way related.

Using Bad Behavior, I have been able to have my stats public on my blog for quite some time and have had no worries regarding referrer spam.

For those who really want to be sure, check out Referrer Karma. Like Bad Behavior (which can be used at the same time as Referrer Karma), RK screens all incoming traffic and those which are giving a referrer are checked. If the site listed as the referrer doesn't have a link to your site (which would be expected), the user is stopped and the attempt is logged for your review.

I used to use it, but if I remember I had issues with some people accessing the board and the admin panel didn't function properly. Things may have changed.

Bad Behavior, however, gets my full endorsement. I just wish someone would develop a phpBB MOD to take advantage of Bad Behavior's database logging and other features that aren't available in the generic interface I mentioned in an above post.

eccerr0r
Registered User
Posts: 20
Joined: Tue Nov 28, 2006 4:51 pm
Location: Colorado, USA
Contact:

Post by eccerr0r »

I'd prefer not to have to deal with it alltogether - but yeah, some way of automatically checking, something like RK sounds like the best way to go.

Just that it's not the first check, you probably need to check *every* time someone sends that referrer link just in case they make it correct for 1 hour... that adds a lot of bandwidth wasted to doing something not very useful... may need to check every so often.

never really is a good, clean way to deal with the problem...

ezlynx
Registered User
Posts: 81
Joined: Tue Sep 10, 2002 8:34 pm

Post by ezlynx »

One very simple mod I used was changing the newest member from the newest registered member to the newest activated registered member. In includes/functions.php, added AND user_active = 1:

Code: Select all

		case 'newestuser':
			$sql = "SELECT user_id, username
				FROM " . USERS_TABLE . "
				WHERE user_id <> " . ANONYMOUS . "
				AND user_active = 1
				ORDER BY user_id DESC
				LIMIT 1";
			break;
And there is no member list. So much work required to have any screwy stuff visible.

romans1423
Registered User
Posts: 1552
Joined: Sat Nov 02, 2002 4:44 pm
Location: Connersville, IN
Name: Rick Beckman
Contact:

Post by romans1423 »

ezlynx: You're fixing the wrong problem. ;)

ezlynx
Registered User
Posts: 81
Joined: Tue Sep 10, 2002 8:34 pm

Post by ezlynx »

Yes. I guess I missed the point (hey, not the first time!). But do find that change handy for a different problem.

Locked

Return to “2.0.x Discussion”