main

Lee Maguire: webslog

Currently:

Grime Scene Investigation

I didn't manage to catch the last couple of episodes of How Clean is Your House? on C4. When the first one aired my mother immediately phoned to demand I watch it. I'm not sure what her message was there: "here is a cautionary example, this is what I've been trying to save you from becoming" or perhaps "here's your chance to experience the thrill of house-keeping criticism vicariously".

Sadly, only the first half of the show features the guilty pleasures a close-up look at the filthy, filthy hovels other people are living in. The second half is the tedious cleaning itself complete with handy tips (90% of which are: "use vinegar").

In a way it reminds me of the characters in CSI. That show features six or seven characters, all of them a variation of TV forensics (and, by extension, apparently all science) genius Dana Scully. Despite the various flaws and idiosyncrasies each character is imbued with (to unique-'em-up a little), there's one trait they all appear to share: they're all Mother-in-Law-style house-keeping snobs.

For example: they'll be in the victim's bathroom, rubbing swabs over everything, bagging bits of brain-matter or whatever, and one of them will say something like "God, when was the last time someone cleaned this shower?".

Maybe it's a common attitude for people who spend their working day sifting through guts and garbage. I guess it's the kind of job that demands some kind of obsessive personality. I've never met a CSI in real-life but certainly the attitude of their fictional counterparts has affected me. When I clean now it's not for myself, not for my mother, not for my guests. I now principally clean for the benefit of people who'll be investigating the unusual circumstances of my death.

So in comparison, what's missing from "How Clean is Your House?" is the dense science-speak and the expensive looking gadgets. Sure, they might send something "back to the lab for analysis" but that's about it. A Biotrace press release invites us to "look out for successful Biotrace products such as the Uni-Lite XCEL luminometer and the company's two rapid surface hygiene tests: Clean-Trace an ATP based test and Pro-tect a colour change protein test." Sorry Biotrace, I must've blinked.

So how about a TV show featuring actual forensics teams examining real homes, ones that aren't crime scenes. A potential nightmare for neurotic cleaners across the country (at least more real than some unlikely chain of events culminating in the Queen requesting the use of their lavatory). Picture a serious looking man in a lab coat inviting you look at a sample of your carpet through a microscope: "Mites. Millions of them...". Imagine men-in-black shining UV torches over your sheets, searching for errant fluids (a la Basic Instinct). Science finding the dirt that normal cleaning hides.

It's Scully in the scullery. Ratings gold, trust me.

(posted 2003-06-19T20:51, )
Apple enforces reverse DNS

Another bit of ammo in the fight for tighter SMTP - apple are rejecting ".mac" mail from servers without reverse DNS. Good for them.

Oddly I know this due to a rejection from a machine that has always had valid reverse DNS. Perhaps they can't cope with RFC2317-style delegation? Certainly there are still a few brain-dead tools still out there that don't.

(posted 2003-06-09T10:54, )
Nice body, shame about the header

If you read my words about fixing SMTP last month you might want to take a look at an draft RFC document I came across regarding technical issues in dealing with spam. As well as being a useful summation of the issues involved, it's also framed (as you might expect of an RFC) in fairly uncontroversial language, which is unusual for a subject for which writers are usually so emotive.

Anyway, one of the things I proposed was rejecting mail without a full complement of completely valid headers. By this I'm referring to a minimal complement of standard mail headers, conforming to their BNF definitions. (Perhaps going a bit further, treating shoulds as musts in the case of things like Message-ID:)

But when it comes to non-standard mail headers... well that's, ahem, a different matter. Maybe 90% of the legitimate mail I receive contains non-standard mail headers. I know that 100% of the mail I send contains at least one non-standard header. And I wouldn't want to encourage the blocking of my mail, would I?

And even in the term "standard" I'm being loose in my definition. I'm referring to any header fully defined in a numbered RFC document, regardless of the status of the RFC itself.

While I moan about the non-standard status of the Precedence: header, I understand why its use is discouraged. For a start, what is it supposed to do? The name itself isn't that instructive. I believe the original (and apparently undefined?) intention was to tag mail so that bulk mail-outs were given a lower transmission priority than normal mail. Then a hacker ( probably Sendmail author Eric Allman) wrote a piece of auto-responder code that checked for the non-existence of this header before sending out auto-responder messages. It soon became the convention of mailing lists managers that this header should be included to suppress these junk replies. And thus Precedence: becomes the de facto standard header to say "I'm a mailing list don't active an autoresponder", the original transmission meaning (I assume) long since lost.

These days if your "out-of-office" auto-responder code needs a simple check to see if any given mail came via a mailing list it should check for something lie the existence of the RFC-defined List-ID: header. Of course, in practice, there's a good chance your mailing-list software or auto-responder predate RFC2919 (March 2001). Welcome to the Internet, if you don't like it file a detailed bug report.

But Precedence: and other non-defined headers were forged in those pioneering days of email. When it was still a experimental. Things are different in the multi-billion dollar Internet of today. After all, we've had twenty years to get our shit together.

Yet, oddly it seems there's still no single standard reference location for standard email headers. Occasionally you'll get a updated document noting common headers but usually if there's a header you need, you'll have to go hunting in the RFC pile until you find it.

When software designers need a custom header many will respectfully use the "X-Something:" naming convention for non-standard headers to avoid clashes with possible future IETF standards. Others, God bless 'em, don't. There is a draft proposal for an IANA-hosted registry of standard message headers. And by the time that happens there are a few non-standard headers I'd like to see standardised:


User-Agent:

There is no standard header for MUA identification. And yet, by default most MUAs will attempt to stamp one on every message they send.

Perhaps there's an air of inevitability about a MUA identifying header becoming standard, and possibly in the choice being User-Agent:. It's recommended and defined in the USEFOR draft for news articles and if that became standard it would be odd not to adopt this precedence for mail. It's already in wide usage in some clients. (My MUA-of-choice for example.)

I don't even know it this has been proposed as part of any mail RFC. If and when it does it'll be a while, possibly a couple of years, before it becomes a numbered RFC. It'll be a few years after that when most people have upgraded to software that uses the new standard. Yep, that's the breathtaking speed of Internet time.

Archive:

Another USEFOR proposal, this one useful for lists. This is essentially the Dejagoo convention of "X-No-Archive:" but without using a double negative to indicate a positive.

Priority:

Technically this a already described in RFC2156 but only for use in X.400 gateways (such as Exchange) rather than in SMTP at large. Essentially this is what "Precedence:" was - an attempt to influence transmission speed. It has three values urgent/normal/non-urgent (non-urgent being less urgent than normal, rather than urgent-but-not-as).

Now given bandwidth has increased at proportionally faster rate than email sizes (at least in my world) you might think that the need to prioritise would already be obsolete. However on machines with fairly hefty mail queues to process the ability to prioritise would be nice - for example on backup MXes after the recovery from an extended primary failure, or on servers processing mailing-lists.

Imagine you have two mailing lists kitten-photos@lists.example.com and timely-financial-news@lists.example.com. kitten-photos usually has pretty large messages and a large subscriber list. You might want to specifically treat "kitten-photos" as "non-urgent" so that it doesn't take priority over normal mail on the system.

To take an example from my recent past, suppose the list timely-financial-news has two types of subscribers: those that have paid you to receive these messages, and those that have not. You want to have at least attempted a delivery to everyone who paid before beginning delivery of those that haven't. It would be nice if all that needed to be done was to mark the paid envelopes with a "Priority: urgent" header. Obviously just having a header won't make mail servers magically able to do this, but rather the idea is to make it obvious what servers that can should expect.

Importance:

This is another X.400 feature and the "human" equivalent of priority (low/normal/high) - a suggestion of the importance of the mail, but with no influence on message transmission whatsoever.

If you only allowed Priority, programmers would probably overload its meaning. Although at college people using the horrible WhiteMail X.400 agent would select "importance: high" assuming that it would mean the message (usually gatewayed into SMTP) would get there quicker. I imagine they're the same people who repeatedly press lit elevator buttons.

An obvious objection to the Priority: and Importance: headers is that they're easy to abuse. I imagine in places where users mark all mail as "high importance" (I don't imagine normal MUAs would need to support Priority:) it becomes meaningless. A typical PEBAC issue.

If high importance becomes normal for you then so be it. I'm more interested in de-prioritising some of my outgoing email. I believe was one of the factors in the success of SMS was that it seemed more polite to send a message that could be replied to at leisure, rather than a phone call that was immediate, urgent. I'm pretty slack when it comes to responding to email, it would be nice if I didn't have to feel I had too. It would also be another little check for auto-responders.

And it's ultimately up to postmasters how their systems send mail. For most email systems that aren't overloaded the "Priority:" header would have little or no effect. The Importance: header is already in use in many systems, but oddly when spammers add it (based a grep of my spam-bin) most appear to set it to "Normal".

I guess if something is consistently abused just becomes another useful heuristic. Certainly I would expect one of those "legitimate email marketeer code-of-conducts" to specify that messages would be marked as low-importance should it ever become widely adopted in internet MUAs (if it isn't already).

Sensitivity:

Sensitivity is another X.400 header and has the possible values "Personal" / "Private" / "Company-Confidential". But perhaps it's not what I'm looking for here. I'd want a standard header that, based on it's value, an MTA could infer routing decisions such as "Don't send this mail over an unencypted link" or "Don't include the message body if you need to send a DSN".

Yeah, in an ideal world sensitive mail would be encrypted.

Mail-Followup-To:

I'm not often a supporter of djb's proposals. I don't support the idea of Mail-Reply-To: becoming a standard because it's just a hack to work around header munging - making it standard would just be compounding the error. I think the real solution for the· Reply-To-munging problem is for people to use smarter MUAs (for example one that checks for a List-Post: header when a reply is requested).

However Mail-Followup-To adds something new and useful to mail by grafting a Usenet convention on to mailing lists. It provides an automated way of saying:

"This message is going to the sos-announce mailing list. Please send followups to the sos-discuss mailing list."
"I'm a subscriber to this mailing list. Please don't send me an extra copy of followups."

or, as implied, (since I believe the normal Usenet follow-up mode isn't to also mail the poster) "I'm not a subscriber to this list, please send me any followups."

Delivered-To:

Another djb-ism. qmail uses it as a form of loop detection. Mailing list managers have a variety of techniques for loop detection but for the simplest auto responder a simple header will usually suffice. There's a lot of software out there using loop detection headers such as X-Beenthere, X-Loop, etc. I imagine Delivered-To:, as well as having a good, descriptive name, is the most widely used for this purpose.

(posted 2003-06-06T20:00, )
copious free time

UI: If you've ever needed to hunt for NTP details when setting up a computer you might want to use pool.ntp.org which, for each query, will give you a random public ntp server using round-robin DNS. Especially useful for roaming laptops.

(posted 2003-06-06T10:05, )