At present, there is a growing nuisance for users and administrators alike of sites that ruin web servers and more particularly, blogs. This nuisance is being referred to as comment, trackback and referrer spams. Various solutions have been proposed with some being applicable to even two of these forms of spam using a single solution.
What is Referral Spam?
A referrer request-header file allows the client to specify the address (URI) of the resource from which the request-URI was obtained. It is a way for an HTTP client to send in the headers, the URI of the page that sent them there. This is especially handy for a site administrator to provide insight as to where the traffic on his web server is coming from. It is also depended upon by the most popular web server log analyzers in providing statistics on the most common referrers.
The HTTP Referrer: header is very useful but it is also completely arbitrary. Any web browser or HTTP client is free to send a forged Referrer: header with any request to a web server. Spammers have taken advantage of the fact that there is no provision for authentication in SMPTP and have used the existing openness to specially craft request with their website in the Referrer: header.
Most people will find it difficult to understand why someone would bother spamming something which only the site administrator will see in the logs. One probable motivation pinpointed is the boosting of search engine ranking. Another is simply to show-up in any stats published by the site. If a site being spammed runs a web server log analyzing software, access to the URL in the top referrer's section is handily obtained by the spammer.
A serious consequence of referrer spam is that the process is often performed via an HTTP "GET" or "POST" request which retrieves the entire body of the document being spammed. A 30k document, for example, will have all the 30k transferred across one's Internet pipe. This results to not a small amount of traffic in the web server which could be very costly since bandwidth is not cheap.
Referrer spam wastes CPU and disk space and can be a source of endless annoyance to server operators. It is being actually fought by search engine developers thus its initial effectiveness in boosting a site's ranking has been considerably lessened. However, the problem persists and much has to be done to conquer it.
Some recommended practices in countering the threat of referral spam include the non-publication of referrers by bloggers, inclusion of the page in robots.txt when referrers have to be published, use of the rel="no follow" attribute and gathering a cleaner list of referrers using JavaScript and beacon images. Some bloggers have begun fighting referrer spammers at the .htaccess level. Others have even taken steps to automate this.
Blocking Users by Referrer Notes
A very useful feature of .htaccess is the ability to block users or sites that originate from a particular domain. When there are tons of referrals from a particular site with no single visible link to one's own site from the said site, the referral probably isn't a legitimate one. The other site is most likely hot linking to certain files such as images, CSS file or other file. The blocking access by referrer in .htaccess requires the help of the Apache module mod rewrite to be able to make out the referrer first. There is a fear that spam would still come in even as .htaccess continue to grow. Blacklisting certain referrers in .htaccess is another option, the effectiveness of which has been greatly diminished due to the ease by which spammers are able to register thousands of domains and rotate them as quickly as they are blacklisted.
The .htaccess generator to prevent people from certain IP addresses, domains or even countries from gaining access to a site or to specific folders can be used. The full IP address has to be typed to block a specific IP. The use of a partial IP address is required to block a range of IPs. Blocking a particular domain can be done by typing the domain without the www. The tail extension is to be typed when blocking a country.
There is no limit to the entries that can be added one at a time. The "add" should be checked after each entry while the generated code is to be copied and posted into a plain text file. This file is then named .htaccess. The "." Before the file name should be noted as well as the absence of any tail extension.
If there is already an .htaccess file in the root of the docs directory or the folder where it is to be applied, the generated code shall be added to the end of the current .htaccess file, taking extra care not to disturb the existing code. It will then be uploaded in ASCII mode.
The rel = "no follow" solution
A coalition of blogging and search engine companies have joined together to support an HTML attribute designed primarily to combat comment spam but have high potentials as well for effective use against referral spam. This attribute is known as the rel ="no follow" is being praised by many bloggers as the ultimate solution for the prevailing problem. The idea is simple enough with the hardest part being the matter of convincing the major players such as Google, Yahoo! and MSN to agree on it.
Tagging a link with rel ='no follow" attribute would prevent any contribution to the site's PageRank. This means that comment and referral spammers will not be rewarded for their illegitimate activities on websites that implement the attribute. The problem gets solved partially but this solution is unable to end it.
This truth is sought to be explained by the fact that it is impossible to reach a 100% adoption thus there will always be an incentive to spam. Spammers essentially do not care whether their techniques are specifically effective as long as they are generally effective. They need no particular reason to hit any site and will do so as their main target is the blogosphere as a whole. It is also quite unfortunate that the resources required to fight spam, particularly referral spam, is far bigger than the resources needed to create it.
Referral spam is an HTTP request. The client doesn't even need to acknowledge the response. All it may need is a simple packet with formatted text.
Spammers take pains to make a request look legitimate. The user - agent string would look very much like MSIE. It used to be that spam came from a single IP but things have definitely gotten more complex since then.
Filtering referrer IPs against spam blacklisting can also be done. Listing the referring URL in any section of a site's web stats should be avoided if the IP is blacklisted. Do not pursue query once a given site is identified as a referral spam host name.
Related Articles
How To Use Linknotes Plugins
By: Danny Wirken | 01/11/2006 | Communication
Communication related articles, learn about VoIP, wireless communication
How To Prevent Comment Spam With Google's No Follow Attribute
By: Danny Wirken | 31/10/2006 | Communication
Communication related articles, learn about VoIP, wireless communication
How To Use Your .htaccess File To Keep Spammers Out
By: Danny Wirken | 31/10/2006 | Communication
Communication related articles, learn about VoIP, wireless communication
How Much Does Your Google Adsense Really Make
By: Danny Wirken | 05/10/2006 | Communication
Communication related articles, learn about VoIP, wireless communication
The Common Goal Of SEO And Paid Search Advertising
By: Danny Wirken | 01/11/2006 | Communication
Communication related articles, learn about VoIP, wireless communication
Riya: A Big Leap In Visual Search Engines
By: Danny Wirken | 16/11/2006 | Communication
Communication related articles, learn about VoIP, wireless communication
What You Should Know About Trackback Spam
By: Danny Wirken | 03/11/2006 | Communication
Communication related articles, learn about VoIP, wireless communication
Learn How To Make Money Online With The Best Free Online Marketing Resources
By: Danny Wirken | 31/10/2006 | Communication
Communication related articles, learn about VoIP, wireless communication
Got a Question? Ask.
Ask the community a question about this article:
Frequently Asked Questions
Former Olympian Yvonne Buschbaum will have sex change
By: muscleguy | 23-11-2007
This is Yvonne Buschbaum, a German track and field athlete who took sixth place in the women's pole vault at the 2000 Sydney Olympics. Buschbaum announced this week that she is retiring from track and field and that she will become a man."I feel as if I am a man and have to live my life in the body of a woman," Buschbaum said in a statement on her Web site. (The statement has been translated from German to English.) "I am aware of the fact that transsexuality is a fringe issue, and I do not want to be responsible for it remaining on the fringe." Although gender reassignment will likely involve Buschbaum taking hormones that are on the World Anti-Doping Agency's list of banned substances, Buschbaum says she has not taken any performance-enhancing drugs during her athletic career. Your thoughts? Do you know of any other athletes that have taken the plunge and had a sex change?
Stalker
By: SimonStudio | 23-11-2007
A person is stalking underage children on a social network and says something controversial and sexual. Multitudes of people block the account. The network disables the account as a result. How does this protect people if one can just login under a different name and email? It would seem it would be in our interest to not disable sick perverted people's accounts, because this would be a way to surveillance their actions. People should be able to block who they want. Disabling an account encourages people to bookmark and write the email down of who they want to harass. The people who bothered to block are now vulnerable. Besides the obvious rejection of the first amendment, I fail to see the logic of kicking people out for what they say, poke or friend. Also if a person has your email address and the network doesn't tell the accused harasser to delete it, how does the accused know a contact is hostile in the first place?
How can I buy a domain?
By: gmester | 21-11-2007
How can I buy a domain?Suppose it is not owned by anyone (I checked it), who do I need to pay? is is for life? how do I know I'm paying to the righrt authority? what doas this payment include?
Has anyone really found good paying jobs online?
By: trismelain | 21-11-2007
Has anyone really found good paying jobs online?
We
By: anatt | 21-11-2007
How to create a web site
What is the maximum size that Yahoo mailbox can ...
By: orpaz_agranov | 21-11-2007
What is the maximum size that Yahoo mailbox can reveive?
Q&A Powered by:
Latest Communication Articles
Telecoms in Pakistan
By: Caroline Telford | 11/10/2008
One of the best ways to measure the growth of a particular country is by the sophistication of the facilities offered by the major telecoms. In an ever-changing industry in terms of technology, it is a must to keep up with the latest that technology has to offer - and this is best done with the help of telecoms.
Telecoms in Brazil
By: Caroline Telford | 11/10/2008
Wireless, landline, mobile, cable, satellite, Internet and Voice Over Internet Protocol or VoIP. These are just some of the types of services offered by telecom providers. No matter which part of the world it is that you are in, the modernity of the telecommunication facilities offered by the major telecom providers in your area will have an impact with the way that you communicate with the rest of the world.
Telecoms in Spain
By: Caroline Telford | 11/10/2008
The impact of not having the means to use modern telecommunication facilities can be illustrated in a few examples. First, if a business owner has two stores which are in separate locations. Without a means for instant communication - the business owner would need to travel from one store to another in order to coordinate the daily aspects of running a business.
Telecoms in China
By: Caroline Telford | 11/10/2008
China is not just the world's most populous country, it is also the third largest country by area. With over a billion people residing in the China, how do you think the country will fare economically and socially if it does not have modern telecommunications facilities?
Telecoms in Canada
By: Caroline Telford | 11/10/2008
When Alexander Graham Bell invented the telephone in 1876, who would have ever thought that the way people communicate would reach the heights that it has today? Far from a typical landline call, there is now video calling, teleconferencing, e-mail, fax, instant messaging, text messaging, mobile phone calls, Internet telephone calls - and all the other forms of communication which made the world a virtually smaller place.
Cordless Phones
By: Pooja Lapasia | 10/10/2008
The cordless phone has become a very essential component in most of the families. In comparison to cell phones, the cordless telephone are said to be much more reliable.
Reverse Phone Lookup - How to Trace Nuisance Prank Calls and Have the Last Laugh!
By: Daniel Jenkins | 09/10/2008
Are you looking for the best way to perform a reverse phone lookup? Read more here.
0870 Number Alternatives
By: Mark Guy | 09/10/2008
An article on the options available to organisations once communications watchdog, Ofcom, implements its planned 0870 changes at the end of 2008.
More from Danny Wirken
Riya: A Big Leap In Visual Search Engines
By: Danny Wirken | 16/11/2006 | Communication
Watch out for new software that will give a new face to search engines. Rather, a program that includes faces in the search function. A new California-based company, Ojos, developed the online photo-based search service named Riya.
Web 2.0, A Guide For Newbies
By: Danny Wirken | 04/11/2006 | Communication
A couple of years back Bill Gates introduce the idea of Convergence to the public. It was a fresh idea that later became a catchphrase for the Internet Industry.
Trackback Spam Explained
By: Danny Wirken | 04/11/2006 | Communication
In most blog applications, there is a feature called Trackback, which allows the user to send a trackback or notification to a different site or another blog that the user referred to in his own blog.
To Blog Or Not To Blog: The Ups And Downs Of Blogging
By: Danny Wirken | 04/11/2006 | Communication
Whenever the subject of the phenomenon called blogging is raised, most people immediately think associated it with an online diary or weblog. The term weblog refers to key words. First is web from the World Wide Web and log, as in keeping a log.
Tips On How To Deal With Anonymous Comment Spam
By: Danny Wirken | 04/11/2006 | Communication
Have you ever experience being flooded with anonymous comments? If yes, then chances are you have been a victim of comment spam. As with everything on the Internet, spam had also evolved. They are no longer limited to email.
The Latest On WordPress Themes
By: Danny Wirken | 03/11/2006 | Communication
As WordPress and blogging become more and more popular, the list of customization options continues to grow. One can attribute that to each user wanting his or her blog to be unique or very much personalized.
The Exciting World Of Video Blogging
By: Danny Wirken | 03/11/2006 | Communication
When the idea of weblogs was first introduce online, it was an instant phenomenon. Suddenly just about everyone feels the need to create their own space online by writing their thoughts. Then podcasting was introduced—blogging in audio form.
What You Newbies Need To Know About Pay Per Click Ads
By: Danny Wirken | 03/11/2006 | Communication
Just about anyone who has been using the Internet in the last few years has no doubt come across the term "pay per click" once or twice. Pay per click is actually one of the less expensive, albeit efficient, forms of advertising online.
Technology 
