27 January, 2004: Pig out

[ Home page | Web log ]

So, the wisdom of Microsoft has visited upon us yet another email worm, which will no doubt please the `anti-virus' software vendors who rely on this sort of rubbish to create a market for their software. Microsoft will presumably squash these people in a little while by flogging its own `anti-virus' product; as an alternative, it could try closing some of the security holes in the Microsoft internet applications, the most severe of which appears to be the way that Windows will guess the type of a file in a web page or email attachment when it's opened, so that an executable program with (say) a .zip extension is presented to the user as a compressed archive but run by the system as an executable.

This latest worm is, apparently, the work of some teenage idiot who's trying to start a denial-of-service attack on the web site of SCO, the internet's favourite litigants. It's nice to see Linux advocates doing their best to make the Free software community look mature. (Alternative conspiracy theories welcome.)

This worm doesn't seem to be too bad, compared to some of the previous ones. Here's the traffic to my personal mailbox: (this is the number of emails having the right size range that were stopped by the spam checker, so it might be a slight overestimate)

Personal mailbox

and on a server which handles mail for a couple of hundred people: (this was done by looking for mails of the proper size range and with no message-ID in the SMTP logs, which seemed the least privacy-invading way to do this; looking at data from before the worm appeared, this seems to give one or two false positives per day, so this plot is probably reasonably accurate)

Whole server

So, this thing pales in comparison to the amount of spam and forged bounces I receive (hundreds per day), and is nothing really to worry about (unless you're still using Microsoft Windows, but it's hardly my place to mourn for those who will do nothing to help themselves).

But Bill Gates is worried, proposing

... some solutions to stop [people] receiving unwanted messages.

and predicting that

In the next 18 months, spam will no longer be a serious problem....

(His statements were made at the Davos shindig for very rich semi-celebrities. Err, I mean `global leaders'. Whatever.)

Gates didn't really say very much, but apparently his death-to-spam scheme will be based on one or several of,

The first and last are designed to stop bulk mail; the second to stop automatic mail. (Oddly, none of them resemble a strategy which turns out -- surprisingly -- to be effective in practice, which is filtering based on the content of the mail. The reason that the effectiveness of content-based filtering is surprising is that, on the face of it, spam is defined by the fact that it's sent automatically in bulk, rather than what it's about. But typically its content contains enough clues for techniques like Paul Graham's to work.)

Although each of the proposed schemes sounds pretty good in principle, they all suffer from a common flaw: there's a lot of legitimate bulk and automatic mail. For instance, every time Amazon dispatches a book, it notifies the customer by email (as must Microsoft properties like Expedia and so forth); Microsoft itself sends out bulletins by email to numerous customers, and email discussion lists distribute messages among thousands of subscribers. Of course, you could imagine a system in which some central server provided a token which could be attached to an email in return for a payment or whatever for recipients could verify -- and which would need to be done only once per message, rather than once per recipient. But that doesn't help much, because a spammer could do exactly the same. And consider the computational load on a system like, say, Hotmail if it had to factor 100-digit numbers every time it sent an email.

Another story suggests that recipients could choose whether or not to charge the sender of a mail, based on whether they judged a mail to be spam. This is a slightly more interesting idea. Presumably the idea is that the sender of a mail would get a token which identifies them as the sender, and put it in their mail. The recipient could test for the presence of this token -- presumably assuming that if it is present the mail is legitimate -- and if it turns out not to be, then the sender's account is debited by an amount set by the recipient. Evidently there would have to be a protocol to discover the fee the recipient has set, since the recipient could arbitrarily decide whether or not to charge.

(Note that this shares the same problems as many of the other schemes, but in a slightly more subtle way. For instance, it can't be used when sending mail to mailing lists, since the sender may have no way of knowing who is on the list and therefore what they may be charged for their mail. Equally, it doesn't help much for Amazon's automated emails. In other respects it has a certain elegance.)

But from Microsoft's perspective this is a great scheme, because it requires a huge and complicated directory of senders and recipients with which their software -- and only their software -- is integrated. They can patent the `technology' and try to stop anybody else from writing software which consults the directory, and then advertise Outlook and Outlook Express as being spam-free email clients. From the perspective of a near-monopoly vendor, making email proprietary is clearly an excellent move. If they think their strategy through properly, they'll be able to make the new scheme compatible with existing email, and build support for it into Microsoft clients and servers, with the intention of convincing desktop users to stick with Windows, and server users to move to Windows.

The obvious response -- to get to market first with an open, non-Microsoft version of a similar idea -- has a number of flaws. The first is that, if they have any sense, Microsoft will patent the technique in use. Nowadays the existence of prior art (or even an existing patent on the same thing) doesn't seem to affect the patentability of an idea, so we can assume that Microsoft won't have any trouble doing so. And the second is that, even if a competing system were up and running, to win any market share it would have to be integrated with Microsoft email products -- which would require Microsoft's consent to do well. (Microsoft Outlook Express doesn't even have a `plug-in' architecture, as I recall, making integrating third-party software with it a total pain.)

Expect more bad news on this front in the future.

Copyright (c) 2004 Chris Lightfoot; available under a Creative Commons License.