Introduction
Farsight Security, Inc.’s Newly Observed Domains (NOD)(tm) feed allows sites to ignore new domains for a few minutes — or even for up to 24 hours — until the Internet’s reputation providers have had a chance to render an opinion on those new domains.
Some spammers may attempt to “race NOD,” trying to use domains virtually instantaneously, before Farsight can update NOD and before customers can retrieve those updates. Fortunately, there’s a technique that can mitigate this spammer strategy: greylisting.
How Does Greylisting Work?
One of the easiest ways to understand greylisting is by comparing it with more polar alternatives:
Blacklisting (also called “block listing”) is an out-and-out negative result: in an email context, if your IP or domain name is blacklisted, your traffic will not be accepted. An example of a popular block list is Spamhaus’ Zen blocklist.
Whitelisting is the polar opposite, a guaranteed “pass” for good traffic. In an email context, this normally means that you’re a highly trusted sender that carefully follows industry consensus best practices, and as a result your traffic is explicitly “pre-approved.” One example of a widely-trusted whitelist is the Spamhaus Whitelist.
Greylisting is an automatic mail processing heuristic that falls “in between” those two extremes. Despite the name (“greylisting”), it doesn’t involve creating a traditional globally maintained list of “grey” domains. Rather, each site programmatically constructs (and automatically expires) its own list.
In a nutshell, greylisting “temporarily rejects” email from new sources with a 4xx status code, effectively saying “Sorry, we can’t accept this message right now, please try again later.” Most “real” mail transfer agents (MTAs) such as Postfix, Exim, etc., will re-queue messages and routinely attempt redelivery several times. Most spambot mailers, however, cannot, or do not bother to do so. Using greylisting is thus an easy way to block a lot of spambot-transmitted unwanted email.
Mailers that do successfully retry and eventually manage to deliver non-spammy email are often automatically added to a locally-maintained list that exempts that site from further greylisting, typically for a locally-determined period of time. This is done to keep greylisting from impacting high-volume but senders that work hard to control spam from their services.
A more in-depth discussion of greylisting can be found in RFC 6647, “Email Greylisting: An Applicability Statement for SMTP,”.
An often-mentioned greylisting implementation is Postgrey, used with Postfix. A list of other greylisting implementations can be found here.
How Does Greylisting Relate to NOD?
Greylisting complements and enhances the effectiveness of NOD. It ensures that even if spammers try to “race NOD,” that attempt will be futile. It represents a second layer of “backup protection,” and protection-in-depth is a fundamental strategy that fosters cyber-security success.
Individual sites may want to experiment, trying greylisting first, then NOD (or vice versa), to see which ordering offers the best spam protection AND the “least impact to legitimate email traffic.”
“What Do You Mean by ‘least impact to legitimate email traffic’?”
NOD intentionally only targets brand-new domains. This means that NOD will never impact well-established domain names. This makes it quite safe for all sites to potentially use.
Greylisting, however, is a heuristic that potentially applies to all domains, new, old, or in-between. As such, it has a greater potential for causing “collateral damage.”
We’ll just consider two examples of this potential collateral damage:
While standards-compliant MTAs (such as Postfix and Exim) routinely (and correctly) handle greylisting, some “custom MTAs” may not.
Greylisting software implementations normally mitigate this issue by
whitelisting known-good senders that are also known to have “flaky” custom MTAs, but such lists are never perfect. Inevitably some obscure non-standards-compliant flaky sites end up being overlooked. This means if you decide to use greylisting, some legitimate mail may potentially end up getting permanently blocked, as if it were spam.
- Greylisting can also introduce potentially-irritating latency. The classic example of this would be trying to use a “password reset” link on a seldom-used site: if you’re urgently trying to reset a totally-forgotten password, bumping into greylisting (and having to wait even fifteen minutes for a password reset message to be retried and eventually be delivered) can feel like an eternity.
We emphasize, however, that these are greylisting “corner cases.” Many sites happily use greylisting and never run into any issues.
Conclusion
As the “spam wars” go on, temporarily blocking newly observed domains with Farsight Security’s NOD blocklist — and potentially complementing that protection with greylisting approaches — can result in excellent anti-spam protection-in-depth.
Joe St. Sauver is a Distributed Research Scientist for Farsight Security, Inc.
