Skip to navigation

malevolent design weblog

This blog is now defunct, but you can find more stuff over at my personal site

Automatic Content Filtering Is ****ing Difficult

I’ve got some PHP & ActionScript code I’ve used to add high score tables to a few games (no links due to confidential subcontracting), and one feature I added at a client’s request was a basic swearing filter with a configurable list of phrases that get removed from any submitted names.

The thing is, you can specify some unambiguously “adult” words/phrases, but you can’t block/substitute most short offensive words without leading to annoying/amusing false positives, as many have found:

When I first started at my previous day job, a couple of colleagues were engaged in an arms race to try to filter a forum and keep it child-friendly, with mischievous users adding symbols or altering spelling, and innocent words caught in the crossfire. The problem was particularly highlighted when “Compe***ions” were announced.

So what’s the answer? Well, usually any automatic blocking needs to be restricted to unmistakeable phrases (limited usefulness), and combined with a system that flags potentially-infringing content but requires manual inspection. Sites with strong user participation can employ additional flagging and reputation systems, but ultimately it’s never realistic to expect to be able to leave user-submitted content unattended.


I've generally used a whitelist when doing this. It's a bit of a cop-out, but it's the simplest way I've found of handling the false matches. I remember Scunthorpe catching me out. Some websites I've seen using simple matching don't like the word 'specialist' as so many spammers are trying to promote a male enhancement drug which is buried in the word.

For anyone really interested in making a good spam filter - the one in 'Slash', the code that runs and some other social news sites, has a very good filter which includes handling ASCII art. It's all in Perl, which is a bit painful for us non-Perl coders, but it's well worth a look just to see all the different types of spam they've had to cover.

Paul Silver, 25th Jun, 1:47pm

Comments are now closed for this entry.