First, why would I like to block the Cuil bot? Based on my experience, Cuil bot is one of the aggressive search engine bots alive, visiting sites from time to time exhausting bandwidth resources and increasing memory usage. It’s not a problem if you have unlimited bandwidth and huge memory resources. But for persons like me who is hosted on Dreamhost PS where I pay by the MB of memory resources, Cuil bot becomes a big problem. When it is around, my memory usage increases to 1GB, slowing this blog down and sometimes killing it.
Increasing my memory resources to 1GB for a few hours is my temporary solution. But why do I need to let Cuil bot hog down my memory resources when I can totally block it? Yeah, Cuil is a search engine, but it’s not sending me any visitors anyway. So, I’ve decided to totally block it. It’s ok if my blogs are not indexed by Cuil. Nobody uses it anyway. Do you? 😀
So, how to block the Cuil bot? Easy. Just add these 2 lines on your robots.txt file.
User-agent: twiceler
Disallow: /
After adding that on your robots.txt file, Cuil bot will never disturb your site again.
Goodbye Cuil bot!
Wow such a memory hog. And the search itself is not very accurate. I tried it and its not so good. Better block it if you are paying for your hosting services. Good thing for people like us using free hosting (blogspot) lol. We don’t have to worry about bandwidth consumption. =)
Never heard of it yet. Though, thanks for the input. 😀
Cui.. what ???
tnx!
-Nhoel of http://www.keywordspeak.com
hi sir macuha! can I ask you a question (although not connected with your post)?
Ive learned from your blog na you use CPAlead. In my case, Im now experiencing probs with them. If you’ll check my site, ang lumalabas sa URL bar ko e about a cpalead-adblock URL. Anu po kaya ang prob? I cant go to the CPAlead website either. Dinisable ko na CPAlead Gateway ko sa blog ko, still, I can’t access my blog. WAH 🙁
thanks in advance sir macuha! any reply/suggestions would help! 😀 Godbless idol! 😀
It sounds new to me, but looks nice though…I tried it few times but gives irrelevant topics. I think this is a new search engine and don’t have image search yet compared to googel.
Thanks for the information Kuya Marhgil…
Good post. Thanks for the info. and I will put it in my note once I updated my site.
It appears that Cuil has ceased to exist:
http://techcrunch.com/2010/09/17/cuil-goes-down-and-we-hear-its-down-for-good/