Content theft and RSS scraping is not going away. As more and more spammers get into the game and the tools they use improve in effectiveness, the problem is only going to get worse. Because of this, RSS feeds are going to remain vulnerable and bloggers will continue to seek out ways to protect them.
However, readers have come to expect and demand full RSS feeds. Some will not subscribe to partial feeds and no one seems to prefer them. Worse still, FeedBurner reports that partial feeds do not generate significantly more clickthroughs than full ones, meaning that there is little business justification for keeping your feeds partial.
It seems that the only justification left for using partial feeds is to reduce the prevalence and impact of content theft. While it certainly does that, the debate about whether or not it is worth the inconvenience it presents to users is a debate that rages on.
Still, many are beginning to take a new look at the partial vs. full feed debate and some who have switched their feed over to partial are thinking about changing it back or making it full for the first time. For those considering the switch, it is important to remember that, while partial feeds are more secure, there are ways to greatly reduce the danger of having a full feed without negatively impacting your users.
All you have to do is take a few simple precautions.
If you are a WordPress user and not on the “.com” free service, you have a slew of plugins available to help you secure your feed detect content theft and even stop it before it starts. If you are considering the switch to a full feed, consider installing the following plugins to help you keep your content away from prying hands.
- AntiLeech: Reviews of AntiLeech range from “better than nothing” to “walks on water”. Antileech works by analyzing the “user-agent” string or IP address of requests for your feed and deliver dummy content to any unwanted visitors. Antileech can also detect new User-Agents by embedding an image into the feed. However, many scrapers hide their user agent to appear legitimate and change IP addresses frequently. Some may even bypass your feed and, instead, scrape the work from third party sources. Still, AntiLeech may be able to block many of your average scrapers before they ever get to your content.
- CopyFeed: A swiss army knife of content protection, CopyFeed can not only add a digital fingerprint to the post, but also embed the IP address of the feed reader and add a copyright statement. It can even ban unwanted IPs from accessing your feed in the future. This enables you to use the fingerprint to detect scraping and then take immediate action against the scraper by banning the IP of the bot, conveniently displayed in the scraped post. One has to be careful though with this plugin not to accidentally ban legitimate readers such as Google Reader.
- Digital Fingerprint Plugin: Though its functionality can be found in CopyFeed, MaxPower’s plugin is extremely simple and plays nicer with FeedBurner. This plugin allows you to embed a digital fingerprint into your post and easily search for it in your WP admin panel. A simple solution for those who do not require all of the power of CopyFeed.
In addition to these plugins, you can also make modifications to your feed directly by editing your template and WordPress core files. Doing that, you can add fingerprints and copyright statements very easily without the need to install plugins.
Free Service Users
If you use a free service such as Wordpress.com or Blogspot, your back is against the wall. You can’t install plugins, you can’t manipulate core files and you don’t have access to the server so there is no simple way to restrict access to your site.
If you are in that situation, you may need to take a serious look at using FeedBurner to control your feed. Though FeedBurner may not be able to do anything to stop your feed from being scraped, it does offer feed modifications and tools that can help you simulate many of the tools available to self-hosted blogs.
First, using their Feed Flare service, you can easily add digital fingerprints and copyright notices to your feeds. These elements will likely be scraped just the same as if they were put there by a WordPress plugin. Scrapers, it seems do not know the difference.
However, more importantly, their “Uncommon Uses” tool allows you to track and follow up on suspicious usage of your feed. Since FeedBurner controls over 1.1 million feeds, they have a pretty clear view as to what is “normal” and “abnormal” usage of one’s content. Though the reliability of this tools has not been proved, one would suspect it to be at least as powerful as any plugin system available.
The drawback to FeedBurner is that they can not stop the scraping of your content once it has started. However, they can at least offer you insight into how your feed is being used and direct you to potential abuse. That is more than most free services provide, which often leave you in the dark.
When it has come to the partial vs. full debate, I have always fallen on the side of full feeds. It has always been my stance that, with reasonable precautions, full feeds can be secure enough to be practical and that they better serve readers and the site.
However, I have never chastised or argued with anyone who has disagreed with me. I’ve always felt that it was a personal decision that every blogger has to make and it comes down to a decision about what role one wants their feed to play in their site.
That being said, it also appears that the mainstream media also disagrees strongly with me on this topic. A recent survey I did of ten of the largest media companies, including the AP, Reuters, CNN and others, found that they all used RSS feeds, but all had their feeds set to either headline-only or to display the first few words.
I’m going to do a more thorough content analysis of the MSM and RSS in a future post.
In the end though, if you are running a partial feed and are considering the switch, there are ways to provide better security for your feed. For, while no full feed will be as secure as a partial one, it can easily be arranged so that the trade-off between security and functionality doesn’t have to be as difficult of a decision to make.
The RSS Icon above is used from Feed Icons under the Mozilla Foundations Guidelines.
I am currently working with a representative from Six Apart to find similar tools for MovableType.