Want to make email easy on your mail servers? Do it the Microsoft IT way. Reject messages from senders that show up on realtime block lists (a.k.a., blacklists, RBLs).

Microsoft IT claims that using RBLs as their first line of defense results in killing 80% of all incoming messages.

You gain other benefits, as well. The processing muscle required for a simple RBL lookup is nothing, so your gateway server can handle tons of messages.

There is, however, a downside.

Innocent users and organizations frequently find themselves on blocklists for a variety of reasons, some of which are flat-out silly.

After extensive and distasteful personal experience with blacklists, the father of Baysian filtering, Paul Graham, noted, “Unlike filters, [blacklists are] run by humans. And humans are all too likely to abuse the kind of power that blacklists embody. Perhaps someone will start another blacklist that tries to avoid such abuses. But how long before that one becomes corrupt too?”

Whether it’s by corruption or incompetence, some RBLs even list all IP addresses designated for use by dial-ups, DSL or cable modems. This effectively knocks out millions of consultants and small business senders.

That’s OK… if you’re Microsoft. They’ll get back to you.

But what if legitimate blocked senders can’t get back to you? After all, the same RBLs that blocked them the first time are still standing at the gateway.

According to Microsoft IT, that’s easy. The sender should call you, so you can add them to your exceptions IP list.

Again, that’s fine for Microsoft.

But it can cause real problems for other operations. Many blocked prospects will simply take their purchases or donations to friendlier climes.

Using RBLs as judge and jury can be deleterious to sales. On the other hand, they can make great consultants if intelligently deployed. When blocklists are considered as a few of the hundreds of components that result in a total spam score, they are helpful and appropriate.

There’s another way to lop off that first wave of spurious senders: Incoming Message Traffic Shaping (IMTS). IMTS on a spam or mail server “shapes” or reduces email message traffic through deferral. The email gateway simply responds to the first message from any unknown sender by telling the sender to try sending again a little later. (IMTS is not to be confused with firewall traffic shaping, which is rarely, if ever, useful for inbound streams.)

Nearly all legitimate email servers are designed to try again… several times, if necessary. But amazingly, few zombies call back… especially those carrying viruses.

When Email Battles tested a spam filtering appliance, before and after activation of Traffic Shaping, the results were revealing.

To avoid test-bench skewing, we installed our trimMail Inbox 631 email gateway (tMI 631) in an active, real world (albeit low traffic) setting in January 2006.

From January through May, about 78,000 SMTP connections a month were attempted. The tMI 631 deferred an average of 42,000 (54%), passing 36,000 messages to other tMI filtering processes.

We turned Traffic Shaping off on the 1st of June. The results were dramatic.

Messages accepted for further onboard processing soared 274% to 98,900.

With IMTS turned off, the number of viruses the tMI 631 had to deal with increased fourfold, from 75 per month to 319. And the average virus size swelled by 1163%.

How Traffic Shaping Differs From Greylisting
The trimMail Inbox’s Traffic Shaping process is different from conventional greylisting, which defers connections based on the sending host’s IP address, the envelope sender address, and the envelope recipient address.Like greylisting, the tMI’s Traffic Shaping module defers first-time SMTP connections from unknown host IP addresses for a configurable time period. Unlike greylisting, the tMI’s deferral process does not consider the envelope sender/receiver address. Instead, it allows the admin to impose longer deferral periods and shorter deferral resets on “spammier” hosts. This means that senders who behave like spammers will find it far more difficult to deliver their payload than they would against a conventional greylisting system.

Spamminess is determined by a number of characteristics of the SMTP conversation, like whether the sender has a valid reverse lookup, the connection originates from a server on a dynamic (DSL, cable, dial-up) connection, the sender trys to send prior to the tMI issuing a HELO, etc.

Spammy IP addresses can also be “throttled,” meaning they’ll only be allowed to send a single message before they’ll be required to negotiate the deferral process again.

In a nutshell, Traffic Shaping puts spammers in the slow lane, and makes it so difficult for them to get their mail through, that most simply give up and bother someone else. Well-behaved senders will find the going smooth after properly responding to the initial SMTP-standard deferral.

Other processes on the tMI gateway were forced to work harder, too. More messages were rejected for bad recipient addresses, or deleted/quarantined/tagged as spam.

So how does IMTS compare against RBLs as an upfront shield? Microsoft IT claims that, by using blocklists, it delivers around 5% of messages attempted. The tMI gateway netted about 6% good mail, with or without IMTS.

But percentages don’t tell the whole story. Microsoft’s spam filtering abilities regularly make the news:

Graham’s position is unambiguous. “If they worked, we’d know by now.”

Assuming that Hotmail Windows Live Mail follows Microsoft IT best practices, you may be forgiven if you opt for a friendlier path.