What can you tell me about the bot Twiceler-0.9?

Dzien Dobry · Post by **Dzien Dobry** » Mon Jun 04, 2007 12:06 am

Should I kill the bot Twiceler-0.9 from http://www.cuill.com/twiceler/robot.html? It sure has been sucking up a lot of bandwidth in the past few days. I googled for Twiceler and searched this forum and found no helpful information. Is this bot just a temporary nuisance or a definite threat? How should it be killed?

Dzien Dobry · Post by **Dzien Dobry** » Mon Jun 04, 2007 11:15 pm

OK. Exactly what code do I put in my robots.txt file to get rid of Twiceler-0.9?

ric323 · Post by **ric323** » Mon Jun 04, 2007 11:19 pm

Some background reading: http://www.robotstxt.org/wc/robots.html

Dzien Dobry · Post by **Dzien Dobry** » Mon Jun 04, 2007 11:58 pm

This is my robots.txt file:

Code: Select all

User-agent: Twiceler-0.9
Disallow: /

User-agent: Googlebot-Image
Disallow: /

User-agent: *
Disallow: /private/
Disallow: /admin/
Disallow: /db/
Disallow: /docs/
Disallow: /cache/
Disallow: /cgi-bin/
Disallow: /files/
Disallow: /images/
Disallow: /includes/
Disallow: /language/
Disallow: /tmp/
Disallow: /templates/
Disallow: /common.php
Disallow: /memberlist.php
Disallow: /config.php
Disallow: /faq.php
Disallow: /groupcp.php
Disallow: /login.php
Disallow: /modcp.php
Disallow: /posting.php
Disallow: /profile.php?mode=viewprofile&u=
Disallow: /profile.php?mode=editprofile
Disallow: /profile.php?mode=email&u=
Disallow: /printview.php
Disallow: /privmsg.php
Disallow: /ranks.php
Disallow: /search.php
Disallow: /viewonline.php
Disallow: /index.php?c=

According to Robots.txt Checker, the syntax is correct but I'm still being inundated by Twiceler-0.9.
My computer log shows ongoing activity by
Agent: Mozilla/5.0 (Twiceler-0.9 http://www.cuill.com/twiceler/robot.html)

Dzien Dobry · Post by **Dzien Dobry** » Tue Jun 05, 2007 12:18 am

The IP address of the bot Twiceler is 38.99.13.126. I tried banning that address from the phpbb control panel. Will that work?

computersOC · Post by **computersOC** » Tue Jun 05, 2007 1:30 am

That won't allow it to see any of your pages (it will just get a banned message).

Dzien Dobry · Post by **Dzien Dobry** » Tue Jun 05, 2007 1:36 am

computersOC wrote:That won't allow it to see any of your pages (it will just get a banned message).

OK. So how do I stop it from sucking up so much bandwidth? Cause it doesn't seem to be respecting the robots.txt file.

ric323 · Post by **ric323** » Tue Jun 05, 2007 1:38 am

Dzien Dobry wrote:
computersOC wrote:That won't allow it to see any of your pages (it will just get a banned message).
OK. So how do I stop it from sucking up so much bandwidth? Cause it doesn't seem to be respecting the robots.txt file.

If it doesn't respect robots.txt, ban it.
Who cares if it can't see your site? If GOOGLE can, people will find you

Dzien Dobry · Post by **Dzien Dobry** » Tue Jun 05, 2007 1:44 am

ric323 wrote:If it doesn't respect robots.txt, ban it.

How do I do that? What's the best way?

ric323 · Post by **ric323** » Tue Jun 05, 2007 2:22 am

IP ban, as you said you have already done.

suisu-jin · Post by **suisu-jin** » Fri Aug 10, 2007 6:30 pm

How can I add Twiceler to the bot list of phpbb3?

Willoughby_wx · Post by **Willoughby_wx** » Sun Aug 12, 2007 3:43 pm

ACP --> Spiders/Robots --> Click 'Add bot' at the bottom --> Name it 'Cuill [Bot]', with Agent match as 'twiceler'.

At the moment i'm just using one of it's IP addresses, 38.99.44.101

blackwatch_uk · Post by **blackwatch_uk** » Tue Aug 28, 2007 4:52 pm

This little bugger has been sitting on my index page all day. I've added him (sorry, it) to the bot list. 'Cuill [Bot]'

Should this be reason to worry? Why is it just sitting on the index page all day?

*Edit* Sorry searched this topic, and just realised I'm in 2.0. I'm running a 3.0 board, sorry.

RMcGirr83 · Post by **RMcGirr83** » Fri Dec 21, 2007 1:18 am

Twiceler aka Cuill uses multiple ips, but I have been reading that you can send an email to costello [at] cuill.com including your url and they will respect your wishes and stop the bot from visiting your site.

I do believe they have about 30 or so ip addresses.

marcusbacus · Post by **marcusbacus** » Tue Dec 25, 2007 4:02 pm

I've just banned the whole IP list that appears in their site (no more than a dozen addresses I guess), but it didn't work I suppose. Today my forum (and the whole domain) went offline due to bandwidth limits and the logs show a zillion hits by them - and I mean a zillion (the log file is 3mb, and it lists just 2 days of hits). I just sent an e-mail to that Costello (properly signed as Abbott

) asking to be removed.