On Why We Don't Use Challenge-Response

I was recently sent an email from one of our users, evidently impressed by a challenge/response mechanism set up by one of our clients on their mail server (even more impressed by the client’s claim that he received “no spam, ever”). He asked us why we didn’t implement something like this.

Hi Luke,

We’ve been keeping an eye on the challenge/response (C-R) debate for quite some time now. I remember we spent a good while debating over whether we should include it in our anti-spam arsenal. After a lot of consideration, I think we’re going to leave it alone for now, and treat it as a “last line of defense”.

A few of the reasons we are choosing not to roll out a C-R solution:

1. Increases the amount of non-legitimate mail traffic. This is actually contrary to the goals of an anti-spam solution.
2. Doesn’t provide as much protection as you’d think. I doubt Eric’s claim of “no junk mail ever”, especially since we regularly get spam emails that are ‘spoofed’ to be from @ourdomain.com addresses.
3. Trivial to work around. Spammers, for all their misdeeds, are inventive, creative little sods. For example, there was a story recently about spammers getting around Yahoo’s automated-account-creation-prevention tool. When you try to create a Yahoo account, you’re given an image with a word on it, which is hard for machines to easily guess. So what the Spammers redirected this image onto their pornography sites. People joining these sites would type in the word they saw, and this would be fed directly into Yahoo. Sneaky, but impressively so.
4. Any kind of automated response will just lead to the auto-responding address being added to the spammer’s list of “active” emails. This results in more spam hitting the address.
5. This, in turn, results in heavier burden on the system.
6. Speaking of which, most spam comes from non-working or false email addresses. A C-R response to each of these could easily result in a DOS attack on our system.

I could go on, but I think you should see by now that there’s a lot to be said AGAINST C-R systems.

However, one of the things we’re keeping a very close eye on for our anti-spam toolkit is the idea of ‘greylisting’ (www.greylisting.org). A brief rundown on the greylisting method:

• Unknown person (john.doe@unknowndomain.com) sends an email to myaddress@ourdomain.com
• ourdomain’s mail server responds with “oops, temporarily unavailable, try again in a minute”
• ourdomain’s mail server notes that it’s got unknowndomain.com’s mail server in its queue of mails
• if unknowndomain.com is a proper mail server, it will wait a couple of minutes and try again
• if unkowndomain.com is using spam software, it will just barf
• unknowndomain.com’s mail server tries sending the mail again, ourdomain.com’s mail server notes that it passed verification, and “whitelists” @unknowndomain.com

It’s like C-R, but without any of the nasty downsides I listed above. One thing I particularly like about this system is that it doesn’t involve any human interaction. My Grandmother could email me and not get confused by the Challenge-Response mechanism.

We’ll probably be testing out greylisting on our secondary mail server soon, and if all goes well, we’ll roll it out onto our primary mail server.

Open-Source groupware

Something that’s come up quite a bit in work recently has been the idea that we need groupware. What we specifically need is:

1. shared address book
2. shared calendar
3. shared mail directories

The third of these, we’ve managed to hack together using courier-imap. Unfortunately for us, most of our users are very reluctant to move away from POP3, so they’re largely unaware of the availability of this really cool technology.

The second of these, we’ve… well… sort of managed to hack together. Using Outlook 2000/XP/2003’s Free-Busy publishing tool, our users publish to a shared folder which they can all read from. I even wrote a nifty little hack for our ‘resources’ (meeting room, projector, etc.). But one of the limitations of the free-busy information is that it doesn’t list the reason for being busy. It wil just say “John is busy from 10am until 11pm tomorrow”, not “John is in a meeting with Jane and Bob from 10am until 11pm in Meeting Room 1”. So again, this gets underused.

Playing about with Mozilla’s Calendar, we have a tool that does what we want, and allows us to share calendars amongst a team. And it does it all using open standards, so we can get in there and hack around it, if we want. Unfortunately, Sunbird (the codename for the calendar software) is still at a very early stage (0.0.2?), and is barely-usable. Definitely not usable within a production environment.

Shared address books? Nightmare. We have a company-wide address book, with all our email addresses in LDAP (one of these days I’ll get around to integrating this with Sendmail, I promise). But we can’t add to this remotely. Perhaps we can, and I just haven’t figured out how yet. Either way, we need something better.

Yesterday, a crazy thought entered my head. Bear with me, because it sounds a little ‘out there’. Would it be possible to remove Outlook completely, and have everyone work through a ’thin-client’ (read: browser-based) solution? We already provide a much-loved web interface to mail (using the horde application framework), so would it be possible to extend this some more? Horde offer a module that lets people browse CVS, which I’m sure will appeal to developers. It also allows them to set up filtering rules and vacation notices and, and, and…

Well, this is useless. Of course I can see the advantages of it. I’m already sold on it. Based on their reluctance to give up POP3, I’m worried about how reluctant they’d be to give up their Outlook. People get remarkably attached to their email.