The crux of the dust-up revolves around false positives… those messages you’re pining for that end out tagged as spam. The Sender Policy Framework council claims Microsoft’s revised Sender-ID Framework causes lots more of them.

First, a bit of background. Both SPF and SIDF are experiments that try to confirm a message sender’s authenticity. When a message is received by a participating system, it “asks” the sender’s DNS server if the individual really exists. The answer ultimately determines the message’s fate.

SPF receivers expect the DNS entries to match either: a message header’s MAIL FROM, or; the sender’s initial connection message (HELO). SIDF receivers look for the Purported Responsible Address. Microsoft claims superiority because, “The PRA is designed to be the entity that most recently caused the message to be delivered.”

As you might expect, that makes PRA a bit more complicated:

The purported responsible address (PRA) of a message is determined by the following algorithm:
  1. Select the first non-empty Resent-Sender header in the message. If no such header is found, continue with step 2. If it is preceded by a non-empty Resent-From header and one or more Received or Return-Path headers occur after said Resent-From header and before the Resent-Sender header, continue with step 2. Otherwise, proceed to step 5.
  2. Select the first non-empty Resent-From header in the message. If a Resent-From header is found, proceed to step 5. Otherwise, continue with step 3.
  3. Select all the non-empty Sender headers in the message. If there are no such headers, continue with step 4. If there is exactly one such header, proceed to step 5. If there is more than one such header, proceed to step 6.
  4. Select all the non-empty From headers in the message. If there is exactly one such header, continue with step 5. Otherwise, proceed to step 6.
  5. A previous step has selected a single header from the message. If that header is malformed (e.g. it appears to contain multiple mailboxes, or the single mailbox is hopelessly malformed, or the single mailbox does not contain a domain name), continue with step 6. Otherwise, return that single mailbox as the Purported Responsible Address.
  6. The message is ill-formed, and it is impossible to determine a Purported Responsible Address.

… And if all else fails, do it the SPF way. At least, that’s the way it was, before the Internet Engineering Task Force consented to a change in the SIDF experiment’s approval. At first glance, it seems to be fairly benign. Sender-ID should consider SPF records to be a replacement for PRA recs if PRA recs don’t exist. So what’s the beef?

Turns out there’s a big difference between “doing it the SPF way” and using the SPF records to do it your way. According to some, it leads to false positves… along with the undoing of SPF’s experiment.

In his appeal on behalf of SPF, Julian Mehnle quotes approval documents for SPF version 2 (a shotgun marriage of SPF and SIDF), which states: “Without explicit approval of the domain owner, checking other identities against SPF version 1 records is NOT RECOMMENDED because there are cases that are known to give incorrect results.”

Further, Mehnle lists cases where legitimate mail could be rejected by a receiver using SIDF methods on classic SPF records:

  • Many mailing lists rewrite the MAIL FROM identity when distributing messages, but do not change the header (PRA) identities. And they are not required to do so by … current IETF standards.
  • Many organizations with their own domains outsource their bulk message sending (newsletters, etc.) to Email Service Providers, who use their own domain in the MAIL FROM identity and the organization’s domain in the From: header, but do not add a Sender: header.
  • If the MAIL FROM is empty (”MAIL FROM:<>“), the MAIL FROM identity, as defined by the SPF specification, falls back to HELO identity while the PRA identity is usually unpredictable.

Mehnle is no shrinking Violet. When he campaigned for a seat as an SPF council member in 2004, he made his intentions absolutely clear. Among other things, he promised, “I plan… to lead and support the SPF project to… explain to the public that SPF Classic is not Sender-ID/PRA. Make no substantial concessions to Microsoft or other political stakeholders against technical reason. Nevertheless do clever PR.”

Further, he added, “I have lobbied against Meng’s decision to merge SPF with Caller-ID into Sender-ID in order to get the 400lbs gorilla AKA Microsoft aboard.”

Apparently, SPF members liked what they heard. But are they willing to live with their decisions?

Harald Tveit Alvestrand worries:

Are you of the opinion that the Internet Engineering Steering Group should try to police which experiments get run on the Internet by refusing to publish RFCs documenting possibly-conflicting experments?

Both of these documents were published at the request of their authors. I know that ways in which they could cause conflict were pointed out to the authors, and that both authors upheld their requests to publish.

If you ask the IESG to assert the power to police this kind of experiment, please make sure you will be happy with the result if they agree with you…”

Others assert that the “do-as-I-please” functionality of the revised SIDF protocol will contaminate both experiments, rendering analysis useless.

Datamonitor’s Kevin Murphy asserts destruction may result from SPF politicking: “Some participants in IETF discussions already suggested that the IESG should publish neither SIDF nor SPF until their differences can be worked out.”

While the fur flies, the Spamroll blog speaks for the victims, who simply want “an authentication standard that is royalty free forever, based on open standards, under a licensing scheme that requires 100% interoperability at all times.”

Meanwhile, spammers… the Always Earliest Adopters… eagerly await any-and-every standard that helps them convince you of their authenticity…