Guests

Get help with installation and running phpBB 3.3.x here. Please do not post bug reports, feature requests, or extension related questions here.
Hervé
Registered User
Posts: 414
Joined: Tue Jun 04, 2019 7:51 am
Location: Belgium
Name: Rudy
Contact:

Re: Guests

Post by Hervé »

P_I wrote: Thu Jul 28, 2022 4:32 pmFor my boards I found the blog post Blocking aggressive Chinese crawlers/scrapers/bots - John Large .co.uk very useful to understand whether or not the guests are legitimate or not. I'm using the solution mentioned in this comment to deal with the automated crawlers/scrapers/bots.
If I understood correctly, this explains how to block some bots.
My question is not to block bots but to understand what all these guests are doing there, maybe some are in fact bots : how to identify the guests who are actually bots and consider them as such ?
User avatar
thecoalman
Community Team Member
Community Team Member
Posts: 5876
Joined: Wed Dec 22, 2004 3:52 am
Location: Pennsylvania, U.S.A.
Contact:

Re: Guests

Post by thecoalman »

You need to research the IP. If it's coming from a hosting company such as OVH and/or cloud service like AWS* it's pretty much guaranteed that it's a scraper. I mention these two because they are commonly associated with such activity. That said they can just as easily be coming from what should be residential services.

The easiest way to identify them is using an analytical tool like Awsats, if you have cPanel it should be available there. Go right to the IP listings and you'll probably find is the top ones are either coming from sources you should not expect traffic from. Note that blocking single IP addresses is only effective very short term.

* You need to be careful about what you are blocking. A common criteria is to block by ASN which can block an entire network for a service like AWS. Duckgogo uses AWS so you need to whitelist their IP's and block the rest if you want to allow them to index your site. As another example if you used OVH hosting you can't block OVH. ;)
“Results! Why, man, I have gotten a lot of results! I have found several thousand things that won’t work.”

Attributed - Thomas Edison
Hervé
Registered User
Posts: 414
Joined: Tue Jun 04, 2019 7:51 am
Location: Belgium
Name: Rudy
Contact:

Re: Guests

Post by Hervé »

Once again I don't want to block anybody.
I would like the bots to be identified as such and not as guests.
User avatar
janus_zonstraal
Registered User
Posts: 6418
Joined: Sat Aug 30, 2014 1:30 pm

Re: Guests

Post by janus_zonstraal »

ACP => general => spiders/bots add bots
if you really want add them, but the only thing the do is eating your server resources.
Sorry! My English is bat ;) !!!
Hervé
Registered User
Posts: 414
Joined: Tue Jun 04, 2019 7:51 am
Location: Belgium
Name: Rudy
Contact:

Re: Guests

Post by Hervé »

How can I know that a guest is in fact a bot ?
Isn't there a way to identify them automatically rather than adding them one by one to a list that will never be exhaustive ?
User avatar
janus_zonstraal
Registered User
Posts: 6418
Joined: Sat Aug 30, 2014 1:30 pm

Re: Guests

Post by janus_zonstraal »

There are maximum 15 bots you have to add.
You find them in "who is Online" => Guests
Sorry! My English is bat ;) !!!
Hervé
Registered User
Posts: 414
Joined: Tue Jun 04, 2019 7:51 am
Location: Belgium
Name: Rudy
Contact:

Re: Guests

Post by Hervé »

I find 4 bots and 2 guests
IP : 136.243.228.177 Mozilla/5.0 (compatible; DataForSeoBot/1.0; +https://dataforseo.com/dataforseo-bot)
IP : 107.174.232.198 Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3393.4 Safari/537.36

How do I know if they are bots and what should I add in
Robot name
Robot style
Robot language
Robot active yes/no
User agent
Robot IP
User avatar
janus_zonstraal
Registered User
Posts: 6418
Joined: Sat Aug 30, 2014 1:30 pm

Re: Guests

Post by janus_zonstraal »

I think the answers are in the "add bot page" where you see the extra info under the settings.
You can also compare with the existed bots in the list.

DataForSeoBot is a bot, the other one is a guest
Sorry! My English is bat ;) !!!
User avatar
david63
Registered User
Posts: 20646
Joined: Thu Dec 19, 2002 8:08 am

Re: Guests

Post by david63 »

You do realise that adding "guests" to the bots group will end up as a never ending process as new bots will visit your board every day (well not quite but there will be new ones frequently visiting) - so unless you sit there monitoring all visits to your board you will always have some bots appearing as guests. There really is no easy way to trap them.
David
Remember: You only know what you know and - you don't know what you don't know!

I now no longer support any of my extensions but they will start to become available here
User avatar
Lumpy Burgertushie
Registered User
Posts: 69224
Joined: Mon May 02, 2005 3:11 am
Contact:

Re: Guests

Post by Lumpy Burgertushie »

I am curious as to why you are so worried about who the guests are.

If it is because you want to figure out how to get them to register then maybe you should show them something
that will make them interested enough. If a gues can only see the first post and not even all of that, then they will not have enough
information to know if they want to register or not.

robert
Premium phpBB 3.3 Styles by PlanetStyles.net

I am pleased to announce that I have completed the first item on my bucket list. I have the bucket.
User avatar
thecoalman
Community Team Member
Community Team Member
Posts: 5876
Joined: Wed Dec 22, 2004 3:52 am
Location: Pennsylvania, U.S.A.
Contact:

Re: Guests

Post by thecoalman »

Hervé wrote: Sat Jul 30, 2022 10:50 am Once again I don't want to block anybody.
I would like the bots to be identified as such and not as guests.
There is two kinds of bots. Those that identify themselves with a user agent which can be added to the bots group. As far as adding them you are going to use unique string in the user agent. For example from the one you posted above next Agent match: put DataForSeoBot.

You need to be careful with the agent match that it's fairly unique. On the other the reason you don't simply use the whole string is some of it can change over time.

The other kind of bots are ones scraping your site or whatever else they are doing. These can't be identified through the user agent because they use common browser user agents. There is no way to handle these through phpBB.

Lastly user agents are not reliable by themselves even for legitimate bots. You can for example use a user agent switcher for your browser and browse your forum as Google. This can actually helpful to see exactly how a bot views your forum. To absolutely identify them you need to do it by IP, Google and other legitimate bots have known range of IP's.
“Results! Why, man, I have gotten a lot of results! I have found several thousand things that won’t work.”

Attributed - Thomas Edison
User avatar
ssl
Registered User
Posts: 1621
Joined: Sat Feb 08, 2020 2:15 pm
Location: Le Lude, Pays de la Loire - France
Name: Fred Rimbert
Contact:

Re: Guests

Post by ssl »

Hervé wrote: Wed Jul 27, 2022 2:59 pm for instance today 152 users : 19 members, 133 guests.
Hi
This is not a permanent observation, just a temporary one, currently the number of guests and bots is much lower.
Would you say that this high number noticed the other day was due to the forum being shared on a social network?
Sorry for my English ... I do my best!

phpBB: 3.3.11 | PHP: 8.2.16
[Kill spam on phpBB] - [Some French translation of extensions]
"Mistress, Mistress someone is bothering me in pm"
Hervé
Registered User
Posts: 414
Joined: Tue Jun 04, 2019 7:51 am
Location: Belgium
Name: Rudy
Contact:

Re: Guests

Post by Hervé »

Hi !
Sorry for the late response.
You are right, the number of visits is counted in the last 5 minutes.
It doesn't change what worries me : since midnight there have been 21 members, 9 bots, 102 guests that is 5 times more guests than members.

I don't understand your question about social networks.
User avatar
warmweer
Jr. Extension Validator
Posts: 11240
Joined: Fri Jul 04, 2003 6:34 am
Location: Van Allen Bel ... gium
Contact:

Re: Guests

Post by warmweer »

Hervé wrote: Tue Aug 09, 2022 12:43 pm Hi !
Sorry for the late response.
You are right, the number of visits is counted in the last 5 minutes.
It doesn't change what worries me : since midnight there have been 21 members, 9 bots, 102 guests that is 5 times more guests than members.
That's perfectly normal and acceptable.
Some of those guests could even be members just having a quick look to see if it's worthwhile (ATM) to log in.
Spelling is freeware, which means you can use it for free.
On the other hand, it is not open source, which means you cannot change it or publish it in a modified form.


Time flies like an arrow, but fruit flies like a banana.
User avatar
HiFiKabin
Community Team Member
Community Team Member
Posts: 6675
Joined: Wed May 14, 2014 9:10 am
Location: Swearing at the PC, UK
Name: James
Contact:

Re: Guests

Post by HiFiKabin »

Several posts split off for clarity
Post Reply

Return to “[3.3.x] Support Forum”