Handling Numeric XML Entities in a Weblog Move

I'm exporting a Radio UserLand weblog to Movable Type for a client, turning Radio's XML archive of weblog entries into a Movable Type import file. I wrote a Java application that employs the XOM XML library to read Radio's weblog data.

Some numeric character entities in Radio's XML data threw me for a loop: â (’), À (¿), Ž (é), ‡ (á) and — (ó). They were transformed -- either by XOM or the Xerces XML parser that it uses -- into garbage characters that display incorrectly in Movable Type.

After fumbling around, I found a solution: Read a weblog entry's XML data as a text file, replace the numeric XML entities with the equivalent numeric HTML entities and parse the resulting file with XOM:

// replace bad character entities with good ones
public void prepareFile(String source) throws IOException {
  File sourceFile = new File(directory + source);
  BufferedReader reader = new BufferedReader(new FileReader(sourceFile));
  File destination = new File("input.xml");
  BufferedWriter writer = new BufferedWriter(new FileWriter(destination));
  String text = "";
  do {
    text = text.replaceAll("â", "’"); // curly single quote mark
    text = text.replaceAll("À", "¿"); // upside down question mark
    text = text.replaceAll("Ž", "é"); // lowercase accented e
    text = text.replaceAll("‡", "á"); // lowercase accented a
    text = text.replaceAll("—", "ó"); // lowercase accented o
    if (!text.equals("")) {
      writer.write(text, 0, text.length());
      writer.newLine();
    }
    text = reader.readLine();
  } while (text != null);
  reader.close();
  writer.close();
}

This is a clumsy solution that relies on escaped markup to produce the HTML entities, but I can't find a better one without editing the client's Radio data by hand. I'm trying to avoid that, because I want to use this application to move other weblogs.

Radio UserLand saves an XML backup of all weblog posts and categories in the software's backups\weblog\Archive subdirectory. If you're using Radio, enable the Archiving in XML preferences to take advantage of this feature, which makes it easier to export the data to another weblog publishing program.

Xml · Radio · Movable Type · Java · Html · 2006/04/08 · 4 COMMENTS · Link

Anna Badkhen's coverage of the East African drought has taken her from Kenya to Somalia, where she filed a report marking 15 years of anarchy and the shooting of a local radio journalist as he traveled to a seminar. The topic of the event: "reducing violence in Somalia."

Politics · 2006/04/07 · 2 COMMENTS · Link

The World's Most Beloved Coroner

I chaperoned a field trip this week to a high school production of The Wizard of Oz, which was fun because kids love getting out of school to see plays. This one had winged attack monkeys roaming the audience, the Wicked Witch singing Michael Jackson's "Bad" and a climactic scene involving water guns.

At the end, they brought a surprise guest on stage: Meinhardt Raabe, the 90-year-old who played the Coroner in the movie.

Raabe, who lives in a retirement community south of Jacksonville, may be the oldest living Munchkin. He's a former Civil Air Patrol pilot during World War II, 30-year Oscar Mayer spokesman and teacher who still makes Oz-related appearances.

Dressed in character, Raabe delivered his famous declaration:

As Coroner I must aver,
I thoroughly examined her ...
and she's not only merely dead,
but really most sincerely dead.

When he finished, several hundred schoolkids roared so loudly I thought it might knock him down.

Movies · Florida · 2006/04/06 · 3 COMMENTS · Link

Netcraft Toolbar Catches Phish

One in 20 people fall for phishing scams and provide their account information to bogus versions of PayPal, EBay and other ecommerce sites, according to a study by Rachna Dhamija of the Harvard Center for Research on Computation and Society.

The study presented real online banking and fake phishing sites to subjects to see if they could tell the two types apart. ... The most sophisticated site caught out 90 percent of the 22 people participating.

I began using the Netcraft Toolbar in October, which works on Internet Explorer and Mozilla Firefox and detects known phishing sites and other suspicious pages. I've tried some of the phish links to see how well the toolbar works, and it has warned me about all of them.

Security · 2006/04/05 · 2 COMMENTS · Link

Movie Downloaders Pay a High Price

Six studios have begun selling movie downloads this week on Movielink. Purchased movies can be kept forever for computer viewing and burned to DVD but can't be watched in DVD players. There's also a limit on the number of computers that can view a movie, and the service and site require Internet Explorer and Windows Media Player.

Prices for new movies are higher than DVDs -- Nicolas Cage's The Weather Man sells for $27 on Movielink and $22 on Amazon.Com. So you're getting less convenience at more cost, though no one had to package, ship or stock the movie.

Though Movielink might be worthwhile for rentals and hard-to-find movies, I'm having trouble seeing the value for major releases. Current customers appear to agree -- aside from The Weather Man, the best-selling downloads are obscure sexually themed movies and foreign films. The No. 1 seller's currently Sappho '68, a "deep, penetrating piece of artful vintage erotica" that rated only one customer review on Amazon.Com:

This is a very poorly made, cheap, nudie film. No story or characters to speak of. It is practically a silent movie ...

Movies · 2006/04/03 · 3 COMMENTS · Link

Mobile developer Russell Beattie likes the new .mobi top-level domain:

... something what we've been desperately needing in the mobile web: A standard navigation scheme. Now users can guess "cnn.mobi" or "yahoo.mobi" or "amazon.mobi" and KNOW that their phone isn't going to barf at them, and the companies will have a standard name to rally around as well. I would imagine that very soon, the handsets themselves will incorporate this, so instead of having to type in .mobi at all, that will be the default navigation scheme like it used to be in browsers for .com.

Domains · 2006/04/01 · 7 COMMENTS · Link

RSS: I'd Rather Switch Than Fight

Jason Young posed a question on Workbench recently:

I have followed the whole RSS soap opera from even well before Mark Pilgrim was writing snarky posts about Winer numbers. I've actually taken more than one opportunity to call it "Internet Jerry Springer" among my IT colleagues and others that I was evangelizing syndication (and RSS) to -- and that was prior to the 2006 events with the Advisory Board and what I feel is inexplicable behavior from Winer.

The whole hullabaloo has the group of people I work with (a National web effort targeting information coming out of the US land-grants and the cooperative extension system) thinking of solely Atom, and where we say RSS -- we really are referring to Atom. Our web apps are going to encourage Atom, produce Atom, consume Atom, etc. Almost entirely after watching the continued craziness surrounding its use.

I admire and respect your work, your tenacity, your continued evangelistic zeal in what you do here, and elsewhere in the things you write about. So I guess this whole rambly comment is to ask - why are you even sticking with this RSS thing? Why not take that energy and put it behind something with what seems far less "Springer Factor"?

This isn't a troll. It's a serious curiousity on my part.

As we work through the long-unresolved issues in Really Simple Syndication and spark up long-smoldering flamewars, people keep asking me why I don't just switch to Atom. The format's an Internet standard that has a well-established framework for resolving disputes, unlike RSS, and the developer community isn't mired in more old grudges than a Van Halen reunion tour.

I have switched.

I chose Atom for a recent programming project and will continue using it exclusively unless the RSS Advisory Board succeeds in clarifying the RSS specification. I'm no longer evangelizing RSS, because I think the problems with podcasting and HTML create huge hassles for implementers and make interoperability impossible.

Not long ago, I spoke with a technology journalist about syndication, and he predicted that a frozen specification will ultimately render RSS a legacy format. He expects Atom to eclipse RSS completely within five years.

That's not a bad outcome. Atom's a nice piece of work created through an exhaustive public process -- Tim Bray said recently that 17,000 messages were posted on its mailing list during development. I don't know Atom well yet as a programmer, but I've been pleased with the results so far.

Though I no longer pimp RSS, I'm still serving on the RSS Advisory Board. A recent post to the group's mailing list demonstrates why. Sean Lyndersay, a member of Microsoft's Internet Explorer team, needs to know the preferred MIME type for RSS documents. He's looking for a recommendation so they can follow the advice and check an item off the team's to-do list.

Like many subjects involving RSS, this is a long-standing issue that lacks resolution because there isn't a clear authority over the format. Mark Nottingham made an effort in 2001 to register an official RSS MIME type, but it failed:

application/rss+xml isn't registered, because the IESG wanted a "stable reference" for the spec (it being in the standards tree). So, it's technically incorrect to use it now; this is one of the reasons this is still a confusing issue.

There needs to be a place where the organizations and individuals with the most invested in RSS, both in financial and personal terms, can come together to resolve matters like this. The board works in the open, favors no vendor and is eager to bring in outside viewpoints. If that remains true, it should eventually gain the trust of the RSS community.

I think it's worthwhile to fight for that, even though I'm no longer interested in fighting for RSS.

Rss · Atom · 2006/03/31 · 33 COMMENTS · Link

Read More Entries