24 Jun 2010

Automatic Content Filtering Is ****ing Difficult

I’ve got some PHP & ActionScript code I’ve used to add high score tables to a few games (no links due to confidential subcontracting), and one feature I added at a client’s request was a basic swearing filter with a configurable list of phrases that get removed from any submitted names.

The thing is, you can specify some unambiguously “adult” words/phrases, but you can’t block/substitute most short offensive words without leading to annoying/amusing false positives, as many have found:

• Scunthorpe problem
• Medireview, reviewuate, etc.
• Tyson Homosexual
• Clbuttic mistake

When I first started at my previous day job, a couple of colleagues were engaged in an arms race to try to filter a forum and keep it child-friendly, with mischievous users adding symbols or altering spelling, and innocent words caught in the crossfire. The problem was particularly highlighted when “Compe***ions” were announced.

So what’s the answer? Well, usually any automatic blocking needs to be restricted to unmistakeable phrases (limited usefulness), and combined with a system that flags potentially-infringing content but requires manual inspection. Sites with strong user participation can employ additional flagging and reputation systems, but ultimately it’s never realistic to expect to be able to leave user-submitted content unattended.