First, why would I like to block the Cuil bot? Based on my experience, Cuil bot is one of the aggressive search engine bots alive, visiting sites from time to time exhausting bandwidth resources and increasing memory usage. It’s not a problem if you have unlimited bandwidth and huge memory resources. But for persons like me who is hosted on Dreamhost PS where I pay by the MB of memory resources, Cuil bot becomes a big problem. When it is around, my memory usage increases to 1GB, slowing this blog down and sometimes killing it.
Increasing my memory resources to 1GB for a few hours is my temporary solution. But why do I need to let Cuil bot hog down my memory resources when I can totally block it? Yeah, Cuil is a search engine, but it’s not sending me any visitors anyway. So, I’ve decided to totally block it. It’s ok if my blogs are not indexed by Cuil. Nobody uses it anyway. Do you?
So, how to block the Cuil bot? Easy. Just add these 2 lines on your robots.txt file.
After adding that on your robots.txt file, Cuil bot will never disturb your site again.
Goodbye Cuil bot!