The Stupid Filter

From the Ideas Whose Time Have Come department:

A team of American scientists are developing the "StupidFilter" - an open-source filter software that will be able to detect "rampant stupidity" of web-content in written English. Similarly to the way spam recognizing software detects suspicious e-mails, the "StupidFilter" will look for pre-fed words or sign combinations that characterize stupidity, assigning particular tokens with different weights based on how often they occur in hand-picked examples of idiotic comments. The developers are using weighted Bayesian analysis along with some rules-based processing, similar to spam detection engines, in order to efficiently distinguish unacceptable messages among the submitted texts.

Their website is here, and no, this is not a joke. It is, however, just about the form of the content, and not the meaning of it. As the FAQ says, it's entirely blind to irony. It's not what you say, it's how you say it.

Aren't you just trying to eliminate comments and discourse that you consider to be stupid?

As much as that might be nice, no. The StupidFilter does not understand, in a meaningful sense, the text that it parses, and our graders select comments that are formally stupid -- that is, their diction, not their content, marks them as stupid. It is not our intent to eliminate debate or disagreement, but rather to programmatically enforce a certain quality of expression. Put another way: The StupidFilter will cheerfully approve an eloquent, properly-capitalized defense of mandatory, state-subsidized rocket-launcher ownership for all schoolchildren.

So you'll still have to deal with that kind of stupidity on your own, though I daresay there's a decent correlation between the type of content this thing will catch and actual pain-inducing stupidity. In other words, it still represents progress, and I intend to hunt down a Movable Type plugin for this when it's ready. Thanks to John for the link.

Posted by Charles Kuffner on December 16, 2007 to Websurfing