Exchange Server 2007 Spam Whitelist Rant

I just blogged this morning about Microsoft Exchange Server 2007 SP1 being released to the MSDN and Technet communities. Exchange Server 2007 is a major upgrade over Exchange Server 2003 with tight integration with Office Communications Server 2007 (OCS 2007), however I have a major complaint. That is - I have been waiting for Microsoft to offer centralized server-based whitelisting challenge-response support in Exchange Server for years and still they haven't added it. I've been on an anti-spam crusade for years. Just check out these posts here, here, here, here, here, and here. Outlook has a client-based whitelist feature, but it is fatally flawed as I wrote about recently.

In case you aren't aware, email whitelists (with challenge-response) enables people in your "allow" list to automatically be routed by your email server directly to your Inbox bypassing all spam filters. If the person is "unknown" a challenge-response email containing a hyperlink with a unique identifier is automatically sent to this unknown person to confirm it's a real person and not a spammer. In cases where you sign-up for newsletters or register on websites with no "live person", you can manually add their domain or their email address to your whitelist.

Now granted, whitelist challenge-response is a guilty-until-proven-innocent scheme that some may find rude, particularly in a business environment where potential customers are looking to do business with your organization. But as spammers continue to find creative ways to bypass keyword filtering by using PDF attachments and image spam, whitelisting will become more popular and even "accepted" as a necessary evil.

One other negative side-effect is that any incoming mail, including spam will generate challenges, which increases the amount of mail that your server has to process. Often spammers forge the "From" address in the mail headers, so the challenges are sent to innocent or non-existent users. If it's a non-existent user, the server will send a bounce-back message which could generate another challenge response resulting in an endless loop. Of course, any challenge-response system should be able to detect System Undeliverable messages and Out of Office messages to prevent these endless loops. Fortunately, many challenge-response solutions are smart enough to filter out many of the addresses whose challenges would be pointless.

One other drawback to challenge-response is if BOTH sides use challenge-response whitelists. You send an email to a friend who sends you back a challenge. However that challenge gets challenged by your challenge-response. Your friend's challenge-response re-challenges your latest new challenge. While the game of one-upmanship challenges continues in an endless loop, neither of you get each other's email. To solve this looping problem the challenge-response system would have to detect that a challenge-response has been already sent to that email address and not send another one until a pre-specified timeout delay has expired - say 1 day. These issues aside, in most cases the advantages of challenge-response outweigh the disadvantages.

There are probably 3rd party solutions that add whitelist support to your Exchange Server, but I doubt they are tightly integrated with Exchange's management interface or Active Directory. So for instance, you couldn't block your sales people from using whitelists, since many organizations would prefer sales people receive some spam rather than turn off potential customers with a whitelist confirmation email. Offering administrative control of whitelist policies is critical to most enterprises. Further, many of the solutions I looked at didn't offer global whitelists. Each person would maintain their own whitelist and an IT administrator couldn't globally allow certain domains or email addresses for the entire enterprise.

Many whitelist solutions I found were hosted solutions where you had to forward your email to the hosted service provider. This requires changing your mail MX record to point to the hosted service provider. The hosted service provider then handles who is in your whitelist and sends out the whitelist confirmation emails. When a person becomes verified the email is then sent down to your email server, typically using POP3. Many organizations are hesitant to have their confidential email go to a third party hosted provider due to security reasons or simply they aren't sure how reliable the provider will be.

There is no reason whitelisting can't be done in-house with some simple software. Essentially all the hosted provider is doing is leveraging a web server, a database of email addresses, SMTP to send email, and POP3. Microsoft Exchange Server already has almost all of these technology pieces. Many Exchange Servers have a web server (i.e. IIS 6.0) for Outlook Web Access (OWA) and almost all have SMTP and POP3 services. The only piece missing is a database to manage whitelists. How hard is that for Microsoft to add?

Microsoft could leverage SQL Server for storing whitelist information or the Exchange Information Store itself. Further, since the emails are being sent through the Exchange Server, the Exchange Server can automatically detect & add the email addresses for any sent emails to the whitelist. Over time, the database for that enterprise's whitelist will become more accurate and negate the need to generate a challenge-response email saving on the email server's resources and bandwidth.

Please Microsoft, I'm begging you - add challenge-response to Exchange Server with a nice centralized admin tool so IT managers can manage corporate-wide whitelists and whitelist policies. Help us to win the war on spam.