7-bit character assassination
There's a good chance NTK now got nuked by a few SpamAssassin installations this week.
NTK usually skirts close to the thresholds many filter systems - its invitation to unsubscribe, WHOLE LINE OF YELLING, the occasional IP addresses in a URL. Every week we get a handful of automatic mails from corporate email systems either accusing it of being spam or asking us to clean up the language and resend (far easier just to unsub the mollycoddled).
But this week we were in DNSBLs.
Not the listserver, that was fine. The machine that was listed was the first link in the chain. The PC the mail was sent from.
The normal focus for DNSBL/RBL schemes (at least historically) is to only be concerned with one IP address: the IP address of the machine connecting to an MX of the recipient's domain, the last of the "untrusted" addresses. This model also assumes that PCs send their outgoing SMTP mail via a "smarthost", usually provided by their ISP. Indeed, sending mail via the smarthost is often recommended as the solution to those adversely affected by DNSBL listings.
This facility isn't of much use to someone receiving mail via POP3, so some client-side mail scanners will pull the relevant IP addresses out of the headers in order to test them. SpamAssassin includes a variety of RCVD_IN_??? tests that are, apparently, applied to all IP addresses found the Received: headers and (I assume) treats them with equal suspicion.
I believe this "overloading" is dangerous when using traditional DNSBLs.
Dave sent out the mail from his home PC this week. His home PC is on a BT ADSL line. IP addresses for his ADSL provider (like many others) are dynamically allocated via DHCP. At some point, probably last week, a new DHCP lease was issued with a new IP address.
One of the previous users of this IP address, back in June, was operating an open proxy (possibly one that had been used for spamming). As a result it found itself on many blacklists. That an open proxy was listed isn't, in itself, a bad thing (although I'm not too sure about lists of open proxies being kept indefinitely without re-testing) since in traditional DNSBL deployment only the listing of a smarthost will be expected to cause issues with legitimate mail.
SpamAssassin, however, provides extra opportunity for false positives that affect single senders, seemingly at random. Every mail, regardless of whether it is sent directly or via a smarthost, will probably contain a header containing the IP address of the PC. If that's a DNSBL listed IP address, then that mail is tainted. And since it only affects a few at a time there's no swell of complaints to the ISP to get things "fixed", just the potential for a few silently dropped mails (more in the case of mailing lists). And, for those that discover the problem before they are assigned a new address, a rude introduction to DNSBLs.
So every mail that got sent out from the NTK list server contained one Received: header containing a tainted IP address, and on some systems this was enough to push it into the threshold of spam (although setups, whitelists and scoring differs, so we can't know how many subscribers this affected.)
Of course there are things that could be done to get around this: pre-process the mail headers to remove the offending headers, send mail from somewhere else, etc. But these don't fix the problem, i.e. the same problem that's probably going to affect a future user of this IP address, and those similarly affected.
And I imagine the address will still be in those DNSBLs for the foreseeable future. I looked into requesting removal on the various websites. Getting it removed from NJABL was painless. DSBL, however, requires that the removal request comes from the listed admin of the address. Their insistence on mail being read by abuse@ and postmaster@ would normally be commendable. But, lets face it, any process that requires mail to abuse@btopenworld.com to be read isn't going to be completed. Especially if it only affects one bemused user at a time.
For anyone who uses dynamically allocated IP addresses my advice is to check it against OpenRBL. You probably won't be able to do anything about it if it is, but at least you'll have a heads up on potential problems.
For anyone using SpamAssassin, or other tools that do retrospective DNSBL lookups - please be aware of this when adjusting the scores.
My simple philosophy is that if your anti-spam filter identifies non-spam as spam, then it's your software that's broken. Modifying mail in an attempt to prevent your software from exposing its own shortcomings would be a disservice.
spam: posted at 19:52, link