Raising the Bar on RSS Feed Quality.

Earlier this week O'Reilly published my latest article "Raising the Bar on RSS Feed Quality." In it I offer recommendations for authoring more useful and effective feeds with an approach that is neutral, practical, and conservative.

This article requires more then just a summary and link though. It was much too involved an effort to not say more.

I am no stranger to publishing having co-produced my own indie music fanzine for several years that eventually made its way onto the web and into the My Netscape Network the started it all. What spurred my interest that eventually lead to this article was a conversation I've highlighted here before.

"You see," lamented Mark Pilgrim, "most RSS Feeds Suck." Mark's comments couldn't have been timelier when first published. I had been experimenting with ways to streamline my intake of weblogs and news using Rael Dornfest's lightweight Perl aggregator blagg. (I now have taken on furthering a RSS feed plugin for MovableType and have other related projects in the works.) While I had achieved a certain level of success, I was surprised and taken aback by the varying quality and inconsistencies of RSS feeds that made my solution less then optimal and at times unreliable. I stopped reading some weblogs because their feeds where too poorly done and simply not useful or worth the hassle. I'm certain I am not the first.

Mark's solution for the technical issues was to develop and publish an "ultra liberal parser" that would allow for common mistakes and other anomalies while processing output.

Joe Gregorio, developer of the Aggie RSS news aggregator, agreed with Mark's assessment, but questioned if Mark's solution was too liberal. "...where is the motivation to fix those feeds?" he asked.

Mark followed with a very noteworthy response: "I. Was. Missing. News." End users don't care about standards. They care about, in this case, getting their news. Developers care about standards because they help developers.

Both viewpoints expressed are valid, important and symbiotic. Without the predictability and structure that standards provide, application developers will struggle to reliably deliver content from feeds to end users.

This exchange only focused on the technical issues of consuming a feed. It does not address other issues that can detract from a feed's utility and effectiveness such as the absence of basic elements or a lack of descriptive and meaningful content that are sadly quite prevalent. (Hence my article.)

Not to sit idly by, Mark along with Sam Ruby and Bill Kearney developed the RSS Validator service that checks RSS feeds for problems and generates friendly and instructive messages to fixing them. The service is optimized for RSS 2.0, but supports other versions of the format. This recent development is significant because it provides a much-needed tool for alerting publishers to issues in their syndication feeds. More work is still needed.

As I mentioned this article was involved. Not necessary the article itself, but all of the discussions, debates and projects it lead me too. In addition to the mt-rssfeed plugin that I completely rewrote, I got involved in the great RSS "war" back in September. I lobbied for RSS 1.0 reform with a simpler RDF-based format and learned a bit about the shortcomings and merits of RDF. I also drafted a "more sane" RSS 2 profile dubbed Extremely Simple Syndication or XSS when it became apparent that was going to be one step forward and two steps back. I also learned about the proper use of XML namespaces in developing my own liberal parser (for the plugin), the shortcoming of the XML::Parser::Lite module (the hard way), using HTTP ETags amd Last-Modified and all about proper XML encoding. Currently I have a plugin for MovableType that does proper XML encoding/decoding (UTF8, CDATA...) nearing release and version 1.1 of the mt-rssfeed plugin on the drawing boards.

Now that the article is finally out after nearly 3 months (it's a long story I won't go into), I'm looking forward to finishing some of these projects off -- at least for a while. I'd like to turn my attention to something a bit different since I'm a technology generalist.

It's been a long strange trip.