July 2010 Archives

In Unicode and the Shavian Alphabet, I wrote about the incompatibility between two online translators: shavian.org's one that translates English into the Shaw alphabet, and Pīnyīn.info's one that translates characters into their Unicode numbers. To summarise: the Shaw alphabet, also known as the Shavian alphabet, was invented in a competition to design an alphabet in which English is spelled as it sounds. I used it as an alien programming language in a cartoon, generating my text with shavian.org's transliterator. I then tried to convert the transliteration into Unicode numbers by pasting into Pīnyīn.info's translator. But the result had the wrong codes, and twice too many of them. Thomas Thurman, author of shavian.org's transliteration script, mailed me to explain why:

With reference to your column at http://www.drdobbs.com/blog/archives/2010/06/unicode_and_the.html : the reason the translator at http://www.pinyin.info/tools/converter/chars2uninumbers.html choked on the Shavian characters you gave it is because all Shavian characters have codepoints above 0xFFFF, and therefore (if you're using UTF-16, which the pinyin.info translator appears to be) they won't fit in a single word and will have to be represented using surrogate pairs. Wikipedia has a reasonable coverage of surrogate pairs: http://en.wikipedia.org/wiki/Surrogate_pair , but briefly, it's a way to represent a Unicode character whose codepoint is too high by using a pair of otherwise illegal characters, both of whose codepoints are low enough. Hence the effect you noted of having "the wrong codes, and twice too many of them".

The fault is presumably with the pinyin.info translator, which shouldn't give out surrogate pairs unless explicitly asked, but it does go to show that, as Wikipedia puts it, "code is often not tested thoroughly with surrogate pairs. This leads to persistent bugs, and potential security holes, even in popular and well-reviewed application software", or as you put it, "computing still is not mature".

Thomas (author of the transliterator script on shavian.org).

How does Google decide what's news? Google "Accounting Error" under "Web", and the spreadsheet joke that I blogged this morning appears as the fourth and fifth search result, under the heading News for "Accounting Error". Google the same words under "News", and it becomes the first entry. I suppose it's because I used the phrase "diplomatic visit" plus some reasonably recent famous names: Mother Teresa, Pol Pot, Milošević; and perhaps even Turing, and Morecambe and Wise (I could, after all, have been writing about intellectual-property rights in the media, or some such). Anyway, Google, I claim my free prize for subverting your news-recognition algorithm.

I also wonder whether the piece you are now reading will itself become news. Could Google–news-spoofing become the new Googlewhack?

Satan is paying a diplomatic visit to St. Peter. They sip tea together in St. Peter's palatial office, and Satan gazes out of the wall-sized window at the cloudscape beyond with its bewinged, behaloed and beharped inhabitants. He spots Joan of Arc, Mother Teresa, Morecambe and Wise, Alan Turing, Tamburlaine —
— "You've got Tamburlaine? He's meant to be one of ours."
— "Head Office computerised our accounting functions. They sacked the Recording Angel and replaced him by a cumulated–sin-score spreadsheet. But the Excel developer missed some cells out of the SUM range. It's fixed now, but no-one noticed the bug for years —"
He gestures, and Satan sees Pol Pot, Genghis Khan, Vlad the Impaler, Milošević, ...

I want to move. But I can't decide where to, for there are problems in my environment, and they seem universal.

Back when Dobbs ran an Artificial Intelligence Newsletter, I wrote an article about how to program Sony's robot dog, the Aibo. Sony were kind enough to send me a CD containing two images of Aibo. I came across this yesterday, and thought you might like to see them. Here is Aibo on his own:

After having dug to a depth of 10 feet last year, New York (USA) scientists found traces of copper wire dating back 100 years and theorised the existence of a telephone network 100 years ago.

Not to be outdone by America, a retired doctor living in London dug to a depth of 20 feet in his garden, and amazingly found traces of copper also. His exploits were published in the Marylebone, Paddington and Pimlico Mercury and the Fulham and Hammersmith Chronicle, both of which proclaimed: "Retired physician finds incontrovertible evidence that British telecommunications predated the War of 1812."

Not to be outdone by a Londoner, Yorkshire made its own contribution to the advancement of archaeological discovery. The county commissioned Charles Sickels, an unemployed excavator, to dig to a depth of 30 feet. He uncovered, in a word, nothing. This prompted the Yorkshire Post and the Huddersfield Daily Examiner both to run separate pieces trumpeting that 300 years ago, Yorkshire had already gone completely wireless.