Fresh from the TekTalk site comes the news why you should stick the deny statement on the attributor.com website attacks.
Attributor acts like a super sleuth for the rich and famous along with those who hold vast copyrights. At least enough to pay their fees. For the fees they charge their clients they claim to rob billions of websites of billions of gigabytes in bandwidth to check and see if they have a word, picture or insight that might be claimed by the client.
Like Tektalk, we found the attributor.com IP hacking our site and in less than 10 minutes had sucked over 25gig in data. Watching the logs these hacks at Attributor suck your site over and over looking at every link to everything over and over again. We know of one video script site that has a search to youtube that got hit for 60 gig and they only had links to Youtube videos. The Attributor spider went nuts and his account was closed.
Now attributor may seem nice but they are taking money from people to rape your bandwidth without cause. Not only that, as a law unto themselves, they do this with no authority, without your permission and without recompense for their rape. Perhaps you think rape is a bit to harsh? If the freaks at Attributor would use a spider and abide by robots.txt they might not do the damage they do. But since they want to see what browsers see they must come into your site like humans, faking you. Robots do not look at videos and graphics and Attributor wants to see everything.
While a tax or fine should be put on Attributor for their brutal assaults on websites we think that you should be aware how to get them out.
Since they do not abide by robots.txt you must deny them in the .htaccess. This can only work if we are all vigilant to changes they will make in this IP due to exposure. We will be watching all our log files to let you know any more we find. Please do the same. Put this line at the top of your .htaccess file at root level
#attributor
deny from 64.41.145.177
How?
Just keep checking your log files for the IP's that download huge amounts of pages and data with the least amount of visits. Look at your logs and see any spikes in activity for a day or two. Get the IP numbers of these offenders and let us know. We can even check and help identify who is hitting you so hard. We all want to know really.
We found this info at Tek Talk
Showing posts with label bandwidth hogs. Show all posts
Showing posts with label bandwidth hogs. Show all posts
Tuesday, February 17, 2009
Friday, May 9, 2008
Cuill Joins The Hog Spotting Limelight
Cuill will steal all your bandwidth its regular robot attacks!
Cuill, pronounced “cool” came to our attention in the last month as it pounded our sites with a single thrust of robots. Grabbing 350 Meg in a few minutes it brought down a site owning a mere 127 pages using a circular and redundant search process. It came on one Drupal site with the archive module active and brought up a page for every day this century and last no matter content or not. If a date is on a calendar it is searched. How many centuries will it go before it turns off?
If it were not for Hog Spotter I would not have figured it out even though the IP numbers were among the highest users Cuill uses a series of IP numbers so that you don’t see one big block coming at you.
The Cuill site is rather sparse with a bragging piece on their 25 million dollar venture capital infusion and a vague reference to important people hired from the real search engines. No names of course but a real home feel to it. Their site claims they are pioneering a new approach to search and that may well be but Hog Spotter wants to put them by the wayside for their rather rude spider.
Cuill sends out their robot spiders all at once and dig through all your links multiple times and it seems from all directions. Similar to the notorious Munax monsters. However Cuill seems to have some kind of spider that rams through your site at high speed. This may be good or this may be bad depending on where you stand.
This is a good thing as the spiders only ram your site for a couple minutes or so. It is a bad thing because even small sites of 120 pages with graphics taking up 1 meg disk space can find the Cuill search engine glomming a huge half gig in bandwidth to get it all before it leaves. In the case of one of our sites the Cuill spider visit meant we were over using system resources and a cutoff in service occurred. Cuill has no form on their site for web owners to recoup the money for the bandwidth or any damage they cause for their abusive programs. So my sparse users and legitimate search engine traffic was cut off from the site for four hours as we restored it.
I said it was a good thing the Cuill spiders only ram your site for a couple minutes? Well that good time wears out fast as the spider returns several times each month. If you can afford to make such gregarious and generous donations to a Silicon Valley startup, as they rip off every site they can find, you can enjoy the hits, as they too will come on as users as well as robots. Fake hits of course, as no person is there, just their spider. When the content is gathered they wont bother you anymore. No, just show your content with some great advertising to keep them in their own loop.
This is just another I wanna catch up with Google, Yahoo and Microsoft at the expense of website owners.
I am not sure where it was written that if you want to get into the search engine game you could ignore rules and propriety and just skuzzy your content in whatever way you can without regard to anyone or thing. But the day of the stealth theft of your content and bandwidth has ended with the rise of the Hog Spotter!
We have not only put a disallow in the robots.txt but have put a deny in the .htaccess to make sure they do not return disguised as browsers
For robots.txt the format to deny them is to place this at the top.
User-agent: twiceler
Disallow:/
Never trust a Hog. To insure their exclusion use .htaccess file. Their claimed IP addresses and code to deny in .htaccess:
#Cuill
deny from 208.36.144.10
deny from 208.36.144.6
deny from 208.36.144.7
deny from 208.36.144.8
deny from 208.36.144.9
deny from 38.99.13.121
deny from 38.99.13.122
deny from 38.99.13.123
deny from 38.99.13.124
deny from 38.99.13.125
deny from 38.99.13.126
deny from 38.99.44.10
deny from 38.99.44.101
deny from 38.99.44.102
deny from 38.99.44.103
deny from 38.99.44.104
deny from 64.1.215.162
deny from 64.1.215.163
deny from 64.1.215.164
deny from 64.1.215.165
deny from 64.1.215.166
For complete protection we suggest blocking their servers at the .htaccess level. As usual check our deny.txt at Tek Talk for regular updates and find out what we use in our websites. Please let us know your experiences and we can add new IPs and search hogs to the lists.
Randy Penn
Cuill, pronounced “cool” came to our attention in the last month as it pounded our sites with a single thrust of robots. Grabbing 350 Meg in a few minutes it brought down a site owning a mere 127 pages using a circular and redundant search process. It came on one Drupal site with the archive module active and brought up a page for every day this century and last no matter content or not. If a date is on a calendar it is searched. How many centuries will it go before it turns off?
If it were not for Hog Spotter I would not have figured it out even though the IP numbers were among the highest users Cuill uses a series of IP numbers so that you don’t see one big block coming at you.
The Cuill site is rather sparse with a bragging piece on their 25 million dollar venture capital infusion and a vague reference to important people hired from the real search engines. No names of course but a real home feel to it. Their site claims they are pioneering a new approach to search and that may well be but Hog Spotter wants to put them by the wayside for their rather rude spider.
Cuill sends out their robot spiders all at once and dig through all your links multiple times and it seems from all directions. Similar to the notorious Munax monsters. However Cuill seems to have some kind of spider that rams through your site at high speed. This may be good or this may be bad depending on where you stand.
This is a good thing as the spiders only ram your site for a couple minutes or so. It is a bad thing because even small sites of 120 pages with graphics taking up 1 meg disk space can find the Cuill search engine glomming a huge half gig in bandwidth to get it all before it leaves. In the case of one of our sites the Cuill spider visit meant we were over using system resources and a cutoff in service occurred. Cuill has no form on their site for web owners to recoup the money for the bandwidth or any damage they cause for their abusive programs. So my sparse users and legitimate search engine traffic was cut off from the site for four hours as we restored it.
I said it was a good thing the Cuill spiders only ram your site for a couple minutes? Well that good time wears out fast as the spider returns several times each month. If you can afford to make such gregarious and generous donations to a Silicon Valley startup, as they rip off every site they can find, you can enjoy the hits, as they too will come on as users as well as robots. Fake hits of course, as no person is there, just their spider. When the content is gathered they wont bother you anymore. No, just show your content with some great advertising to keep them in their own loop.
This is just another I wanna catch up with Google, Yahoo and Microsoft at the expense of website owners.
I am not sure where it was written that if you want to get into the search engine game you could ignore rules and propriety and just skuzzy your content in whatever way you can without regard to anyone or thing. But the day of the stealth theft of your content and bandwidth has ended with the rise of the Hog Spotter!
We have not only put a disallow in the robots.txt but have put a deny in the .htaccess to make sure they do not return disguised as browsers
For robots.txt the format to deny them is to place this at the top.
User-agent: twiceler
Disallow:/
Never trust a Hog. To insure their exclusion use .htaccess file. Their claimed IP addresses and code to deny in .htaccess:
#Cuill
deny from 208.36.144.10
deny from 208.36.144.6
deny from 208.36.144.7
deny from 208.36.144.8
deny from 208.36.144.9
deny from 38.99.13.121
deny from 38.99.13.122
deny from 38.99.13.123
deny from 38.99.13.124
deny from 38.99.13.125
deny from 38.99.13.126
deny from 38.99.44.10
deny from 38.99.44.101
deny from 38.99.44.102
deny from 38.99.44.103
deny from 38.99.44.104
deny from 64.1.215.162
deny from 64.1.215.163
deny from 64.1.215.164
deny from 64.1.215.165
deny from 64.1.215.166
For complete protection we suggest blocking their servers at the .htaccess level. As usual check our deny.txt at Tek Talk for regular updates and find out what we use in our websites. Please let us know your experiences and we can add new IPs and search hogs to the lists.
Randy Penn
Subscribe to:
Posts (Atom)
