Saving Bandwidth on RSS Feed Details

With the current interest in rssCloud and PubSubHubbub (PuSH), I've been thinking about all the bandwidth that's consumed by the RSS elements that describe the feed. When a client requests an RSS feed 10 times in one day, it gets the basic details of the feed over and over again. When clients request the Workbench feed, they get 1,800 characters containing optional RSS elements that I haven't changed in years, except for the PuSH element I added last month. Workbench has 1,900 feed subscribers, so if they average 10 checks a day, they're consuming 32 megabytes every day on information they know already.

James Holderness directed me to RFC3229+feed, a method to request partial RSS feeds that omit elements that a client has already seen. That's useful and has been adopted by some feed publishers and clients, but as far as I can determine, the approach still sends all of the channel elements that describe the feed itself. I wanted to float an idea here to see if it would be useful:

<rssboard:feedDetails>
  http://ekzemplo.com/feedinfo.rss
</rssboard:feedDetails>

This channel-level RSS element identifies a URL that contains the full details about the feed. The details would be expressed as an RSS feed without any item elements.

An optional ttl attribute could contain the number of days the publisher would like clients to cache the information before checking it again:

<rssboard:feedDetails ttl="30">
  http://ekzemplo.com/feedinfo.rss
</rssboard:feedDetails>

A feed publisher who wished to make use of this could move all channel elements except for title, link, description and atom:link to the detail URL. Title, link and description are required in RSS, and atom:link identifies the feed's URL so it can't be moved.

Rss · Programming · 2009/10/13 · 6 COMMENTS · Link

World's Oldest Man: 'Life is Short'

Walter Breuning, the world's oldest man at 113 years, gave a pretty amazing birthday speech Monday at the Montana retirement home where he lives:

Life begins each morning whether we have succeeded or failed or just muddled along. Life is a school to learn, not to unlearn.

Life is the creation by God and if you would know God, be not a solver of riddles. Look about you and you shall see him playing with your children. Look into the air and you shall see him walking in the clouds, out-stretching his arms in the lightning and descending in rain. You shall see him smiling in flowers, then rising and waving his hands in trees.

Life is a great teacher of truth. What is truth to one is not truth to another. What is true in one country may be false in another.

Life is short but the influences of what we do or say is immortal. There needs to be much more of the spirit of fellowship among us and more forgiveness. The power of gentleness is little seen in the world.

Remember that life's length is not measured by its hours and days, but by that which we have done therein. A useless life is short if it lasts a century.

There are greater and better things in us all if we would find them out. There will always be in this world -- wrongs. No wrong is really successful.

The day will come when light and truth and the just and the good shall be victorious and wrong as evil will be no more forever.

Life itself teaches us to best prepare for that future which we hope for and for that journey to that land un-known, not made by hands.

Everything just is beautiful; everything beautiful ought to be just.

The mystery of the world remains unknown. Our maker alone is the key which unlocks all the mysteries of the universe.

The world is neither a prison nor a palace of ease, but rather for instruction and discipline.

This world has been good to all of us.

Aging · History · Philosophy · 2009/09/25 · 2 COMMENTS · Link

PubSubHubbub is a Lot Easier Than It Sounds

I've begun digging into PubSubHubbub (PuSH), the real-time RSS update protocol created by Brad Fitzpatrick and Brett Slatkin of Google and Martin Atkins of Six Apart. I was under the impression that it's harder for RSS publishers to use than the RSSCloud Interface, but that isn't the case. The specification is simple and precisely written, adopting conventions like RFC 2119 that make a spec considerably easier to understand, and it communicates using basic HTTP requests.

I wrote the software that runs the Drudge Retort, so I decided to add PuSH support to it this morning to see how it works. PuSH delegates all the work required for update notification to a server called a hub. Google offers a hub at http://pubsubhubbub.appspot.com/ that's free for use by all feed publishers, so I'm relying on it.

First, I added a link element to the Retort's RSS feed that identifies the feed's update hub:

<atom:link rel="hub" href="http://pubsubhubbub.appspot.com" />

Because this element comes from the Atom namespace, I had to make sure it was declared in the feed's top-level RSS element:

<rss version="2.0"
  xmlns:dc="http://purl.org/dc/elements/1.1/"
  xmlns:atom="http://www.w3.org/2005/Atom"
  xmlns:sitemap="http://www.sitemaps.org/schemas/sitemap/0.9">

The bold portion is the Atom declaration. I already was using an Atom element in the feed, so I didn't need to change this.

When a new story is posted on the Retort, the PuSH hub must be notified that a change has occured. This is handled by sending a ping to the hub with the URL of one or more feeds that have been updated.

I've written an open source Weblog Pinger library in PHP, so I upgraded it to support these pings. A PuSH ping employs HTTP requests (REST) instead of XML-RPC, the protocol used by Weblogs.Com and similar services. I wrote a new function, ping_rest(), that can send a ping to any PuSH server.

By the time I was done, I'd spent an hour on the code and a few hours testing it out. So now when I post a new item on the Retort, Google's PuSH server sends the full text of the item to all readers that support the protocol. This is faster and simpler than RSSCloud, which tells readers to request the feed again.

To give you an idea of how fast PuSH can be, when I posted a new story on the Retort, it showed up 20 seconds later on FeedBurner, one of the first RSS services to support the protocol.

Rss · Programming · Notification · 2009/09/17 · 12 COMMENTS · Link

RSSCloud Should Not Be Controlled by One Person

I posted a call for comments last night on RSS-Public, the mailing list of the RSS Advisory Board, asking what people think the board should do in response to the ongoing effort to revise the RSSCloud Interface.

The interface has been a part of the RSS specification since the publication of RSS 0.92 in December 2000. It determines how software can use the cloud element in an RSS feed to connect to a server that offers real-time notifications when the feed has been updated. In a nutshell, here's how it works:

• A user subscribes to an RSS feed that has a cloud element which identifies its cloud server.
• The user's RSS reader contacts the cloud server, asking to be notified when the feed is updated.
• When the feed has been updated, the software publishing the feed sends a ping to the cloud server.
• The cloud server sends a notification to the IP address of all RSS readers that asked for updates.
• The RSS readers immediately request the feed.

Cloud communications can be sent using XML-RPC, SOAP or REST aside from pings, which are sent using XML-RPC.

Dave Winer recently began an effort to revise RSSCloud, persuading WordPress founder Matt Mullenweg to adopt the still-in-progress proposal on all 7.5 million blogs hosted on WordPress.Com. Winer has made three significant changes to the interface.

First, he changed the fifth parameter of a notification request on the REST interface to a series of named url parameters (url1, url2, and so on upwards), each containing the URL of a feed monitored by the cloud.

Next, he added a new ping format to contact cloud servers using REST.

Finally, he has proposed adding a sixth parameter to the notification request, but only for REST requests. The sixth parameter, called domain, identifies a server that will receive notification updates from the cloud server. It's an alternative to using the IP address for notifications.

Winer, the lead author of several versions of the RSS specification and one of the best-known authorities on syndication, is making these changes unilaterally.

Because RSSCloud has been a part of RSS for nine years, I thought it wise for the board to decide what, if anything, it should do regarding this effort. My personal belief is that it's extremely unwise to give a single developer the authority to revise this interface and author its specification.

Ideally, a group should decide what changes should be made to the next version of RSSCloud. This group could be the RSS Advisory Board, which deliberates in public and has 10 members from across the RSS development community, or it could be an ad-hoc group formed strictly to work on the effort.

As a member of the board for five years, I've had a lot of experience dealing with the consequences of a specification process that is closed to public participation and drafted with imprecise language. It leads to situations like the long-running battle over the enclosure element, which carries podcasting files and other multimedia over RSS. As described in the board's RSS Best Practices Profile, the RSS specification doesn't make clear whether an item can contain more than one enclosure. Developers disagree over what the specification means, so interoperability suffers as some allow more than one enclosure and others don't.

I realize that I'm tilting at windmills to suggest that Winer let the RSS Advisory Board get anywhere near the effort. Jon and Kate have a better chance of getting together. But as developers such as Mullenweg implement RSSCloud, they should insist that the revision process take place in public and involve a group of software developers and feed publishers who have the power to approve or reject each change. The group should write the specification together.

Letting Winer make all the decisions by fiat will just buy years of arguments over what his spec means and why no one should ever be allowed to change it.

Related post:

• According to PubSubHubbub developer Brad Fitzpatrick, Winer has said that RssCloud "is frozen and a 'done deal'."

Rss · Programming · Pubsub · 2009/09/15 · 9 COMMENTS · Link

World's Oldest Person Dies, Takes 1894 With Her

Gertude Baines died yesterday at the age of 115 years and 148 days. She was the world's oldest person and the 16th oldest ever, according to a cool table prepared by longevity geeks on Wikipedia. Baines, born April 6, 1894, was the grandchild of slaves who worked as a maid at Ohio State University dorms until her retirement to a Los Angeles nursing home a decade ago. She was a non-drinker and non-smoker who once told a reporter, "I did not have a lot of fun as an adult but I enjoyed going to church every Sunday." She was hoping to live long enough to vote for President Obama again in 2012.

Upon her death, the oldest known person becomes Kama Chinen of Okinawa, Japan, an island that has the highest percentage of centenarians in the world. Sixty-seven of every 100,000 Okinawans is over 100.

The line of oblivion, the starting date for the living history of the world, moves forward 13 months to Chinen's birthday on May 10, 1895.

There's no longer a person around who could have witnessed the revival of the Olympics after 1,500 years (June 23, 1894), crossed the newly opened Tower Bridge in London (June 30, 1894), or lived through the First Sino Japanese War (August 1, 1894 to April 17, 1895). There also are no fans of the soccer team Manchester City FC who can say they've followed the Blues since day one (April 16, 1894).

Baines was the last person who could have hung out with assassinated French President Marie Francois Sadi Carnot (died June 25, 1894), the author Robert Louis Stevenson (Dec. 3, 1894) or Charles Frederick Worth, the British fashion designer who created Haute Couture (March 10, 1895).

Her lifespan reached from the death of abolitionist Frederick Douglass on Feb. 20, 1895, to the inauguration of the first black president on Jan. 20, 2009.

Writing this post gave me a theory about how sports columnist Mark Whicker could have penned that odious Jaycee Dugard column while being blind to its insensitivity. Whicker, a columnist for more than 22 years, writes 200 columns a year. That's an enormous word beast to feed, and you can't fill it up without developing some odd predilections that give you something to write about when the inspiration cupboard is bare. Whicker used the same gimmick in a 1991 column about the release of journalist-turned-hostage Terry Anderson in Lebanon, so he's a guy who obsesses over the sporting events that people miss when they're kidnapped.

As someone who obsesses over the merciless march of time and has filled Workbench with 2,792 posts over almost 10 years, I can to a degree sympathize.

Aging · History · Oblivion · 2009/09/12 · 6 COMMENTS · Link

ESPN Spikes Rob Neyer's Post about Mark Whicker

The web site for ESPN spiked a blog post today by acclaimed baseball writer Rob Neyer in which he appeared to call for Orange County Register sports columnist Mark Whicker to be fired for his widely criticized column making light of Jaycee Dugard's 17 years of captivity at the hands of a sexual predator.

The blog post was submitted at 08:57:40 PST and had been deleted from ESPN's web site by the afternoon, but it was cached by several search engines and could be recreated, aside from some text at the end.

Neyer explained the removal in an email Saturday morning. "At ESPN.com, we have a policy prohibiting media criticism, and I ran afoul of that policy, however unintentional," he said. "Thus, an editor pulled the post from the site. Considering our policy, I could hardly complain."

Here's the text of Neyer's spiked post:

You've probably already read or heard something about maybe the worst sports column, ever. If not, Shysterball's take is a good start (among many, many possibilities).

About all this, my friend Keith Scherer writes:

"Rob, You know I work in criminal law. I deal with murder and molestation and all kinds of human depravity every day, so I have a strong stomach. It takes a lot to repulse me at this point, but the lynch mob that went after Mark Whicker this week made me sick. Worst post ever? Really? Can we not communicate without hyperbole anymore?

"The guy misfired. He used a silly hook to bracket a dull article that his editor should have canned. Maybe it was stupid and maybe it was offensive, but the guy didn't mean any harm, and it's disingenuous for his attackers to imply that he's indifferent to child abduction and rape.

"He and his editor deserved a scolding, perhaps, but to go after him they way the mob's doing it – writing letters, trying to get him fired, publicly humiliating him – is loathsome. It's America at our self-aggrandizing, self-righteous, politically correct worst.

"I don't know anything about this guy, but I'll assume -- because this isn't only about Whicker -- he's like most middle-aged white guys. I'll assume he has a family that loves him and can't understand the hatred being directed at him. He's in a dying profession and I'll assume he's very well aware of it, is scared to death he's going to be out of work soon, with no way to pay his mortgage, pay for groceries, or pay for health insurance for his wife and kids. I'll assume he (like many of us) has few, if any, transferable skills, so that every day is a catastrophe just waiting to happen. His life, like many lives, is hard enough already without a posse of strangers trying to kick him to the curb for a lapse of taste.

"Again, this guy didn't mean any harm. He was careless, thoughtless (it's not the first time he has used this kind of hook), and the article really didn't need to be written in the first place, but he's not a moral monster.

"You know me, and you know I spent years putting child sex offenders in jail. I'm not insensitive to the trauma this child and her family went through, and I'd be among the last to trivialize it. But people need to get a grip. What trivializes trauma is when people put on airs and imagine that they themselves are the victims of things that happen to people they never met.

"To call for his head over this -- that's morally wicked. It seems to me, at this point, he's more sinned against than sinning. I hope his family members have forgiveness in their hearts.

"Keith"

I'm not nearly as charitable as Keith (which is just one of my many failings). But while I agree that it's morally wicked to call for Whicker's head because he has failed, morally, what about calling for his head because he's incompetent?

It's especially easy for me, because I'm infatuated with good writing and I abhor bad writing, and columns like Whicker's give my profession a bad name.

We might reasonable assume that it's the worst Whicker can offer ... but are his best efforts better enough, and frequent enough? Again, I don't know. Nor am I going to know. Only his readers can know. And his editors, who should know him better than anyone. Speaking of whom, Keith Olbermann has reserved his enmity for those editors (here's video; relevant clip begins one minute in, if Olbermann's politics ...

Keith Scherer is a criminal trial attorney and baseball writer who contributed to Neyer's book Rob Neyer’s Big Book of Baseball Blunders.

After Mark Whicker's column drew furious criticism from hundreds of readers, bloggers and Twitter users around the globe, he publicly apologized and the newspaper's deputy editor for sports, John Fabris, wrote an apology on behalf of the editors who were responsible for its publication.

In a phone interview with Mallary Jean Tenore of the journalism site Poynter Online that followed the publication of his apology, Whicker continued to maintain that he did not make light of Dugard's tragic story.

"I vehemently believe I wasn't insensitive about the fact that she was kidnapped," he told Tenore. "I never made light about the fact that this woman was abducted. I don't think anyone can cite anything in the column that says I did."

On Thursday, former sports journalist Keith Olbermann called out Orange County Register sports editors David Bean, Todd Harmonson and Keith Sharon during the "Worst Person in the World" segment for green-lighting Whicker's column.

Olbermann said, "At least once a career, no matter how good, every columnist, every commentator, will write something so bad, so inappropriate the editors just have to kill it. Mr. Bean, Mr. Harmonson and Mr. Sharon failed to do this. The usually thoughtful Mark Whicker, 22 years at that paper, went tone deaf. ... Mark Whicker will take his lumps for this, deservedly so ... But you're the guys reading this deciding whether or not it gets published, and you say 'great'? You are in over your head, gentlemen."

Update: An earlier version of this story made it appear that Whicker said the "fast-moving, quick-to-judge culture of the web" was to blame for the reaction to his column. Those words were Tenore paraphrasing her interview with Whicker, not a phrase he used. I apologize for the error.

Sports · Journalism · Crime · 2009/09/12 · 6 COMMENTS · Link

Sportswriter Mark Whicker Makes Light of Dugard Kidnapping

On Tuesday, Orange County Register sportswriter Mark Whicker used Jaycee Dugard's 17 years of captivity at the hands of a sexual predator as a premise for a light-hearted sports column. The result may be the most astonishingly tasteless thing I've ever read in a newspaper. Here's how Whicker starts:

It doesn't sound as if Jaycee Dugard got to see a sports page.

Box scores were not available to her from June 10, 1991 until Aug. 31 of this year.

She never saw a highlight. Never got to the ballpark for Beach Towel Night. Probably hasn't high-fived in a while.

She was not allowed to spike a volleyball. Or pitch a softball. Or smack a forehand down the line. Or run in a 5-footer for double bogey.

Now, that's deprivation. ...

The column runs down the sports events, figures and trends that Dugard might have missed while she was being held captive, repeatedly raped as a young child and gave birth to two children by her abuser. Whicker ends with a play on words about her escape:

And ballplayers, who always invent the slang no matter what ESPN would have you believe, came up with an expression for a home run that you might appreciate.

Congratulations, Jaycee. You left the yard.

In the 48 hours since publication, the column has drawn widespread outrage from bloggers, Twitter users and readers of his newspaper.

"This is quite possibly the worst sports column ever written," Matt Welch declared on Reason.

"I can't decide what's worse about this column, the premise or the kicker at the end. The thing is, Mark Whicker is one of the most underrated columnists in the country. Not sure what he was thinking with this one," Boston Globe sportswriter Chad Finn wrote on Twitter.

Greg Simons, a commenter to the weblog Shysterball, claimed he got a response from Whicker to his email complaint. Simons called the column the "most revolting hook I've ever read" and asked Whicker if his next column would be about 9/11.

According to Simons, Whicker responded, "The revolting thing is that you would equate a column that celebrates the release of Jaycee Dugard, and tries to put the length of her 18-year kidnapping in a context that everyone can understand, with a terrorist attack that killed 3,000 people. And then you draw a value judgment about me based on such a preposterous parallel."

Whicker, a sportswriter for more than 27 years and a longtime columnist at the Register, describes himself as a "wary refugee in tech-land" on his Twitter account mwhicker, where his updates are protected from view.

Jason Fry, author of the Reinventing the Newsroom blog, says of the column, "Sometimes we all need to be told, 'This isn't running. One day you'll thank me.'"

Update: Whicker has apologized.

Sports · Journalism · Crime · 2009/09/09 · 2 COMMENTS · Link

Read More Entries