Dealing with junk mail

-----BEGIN PGP SIGNED MESSAGE-----

[ To: cypherpunks ## 09/17/96 09:26 am ##
  Subject: Dealing with junk mail ]

There has been a bunch of stuff in the news lately about junk
e-mail, including the recent judge's ruling that AOL must allow
known junk-mail through to its subscribers while the judge hears
arguments from AOL and the junk-mailers.

I'm a little divided on the whole issue of whether or not it's wise
policy for ISPs to, in general, refuse to deliver suspected junk
mail.  The obvious problem is that it puts ISPs in the position of
deciding whether or not some piece of e-mail is worthwhile.  (To
clarify, I certainly *don't* think that ISPs should be prohibited by
law from blocking delivery of or access to anything they choose
to--there are plenty of ISPs to choose from, and users can move if
they don't like their current ISPs' policies.  I just don't think
I'd like Delphi to start filtering my e-mail without asking me for
permission and instructions first.)

I've been thinking about an alternative approach to blocking
commercial spam.  It has some potential technical problems, but I
think it could be made into a workable 95% or so filter.

Many people are already filtering messages locally, and now some
providers are getting into the act, as well.  Unfortunately, because
of the economics of junk e-mail, I think that this, by itself, will
probably lead to people refusing to accept almost all e-mail from
people they don't know.  This is a really bad outcome.

What I'm proposing is an extension to this, in which many peoples'
filters coordinate their actions to detect and block spam.

Each user has a mail filter with a set of rules written either by or
for that user.  The mail filter does one of four things with each
piece of e-mail it receives:

a.   It lets the e-mail through immediately.  (E-mail from friends,
employers, employees, family members, etc. would probably be in this
category.)
b.   It discards the e-mail immediately.  (E-mail from people you
really didn't like, and from people who have spammed you in the past
would probably be in this category.)
c.   It puts the e-mail on hold in some storage area.
d.   Send e-mail back to the sender, informing him of conditions
     under which the user is willing to accept this e-mail.  This
     might be things like requiring anonymous users to provide some
     minimal kind of identity, or telling senders ``I'll read your
     e-mail for one dollar in digicash,'' or ``I'll read your e-mail
     if you carry out this computationally expensive calculation, or
     some other thing.

For e-mail in the third category, some kind of summary report is
sometimes generated, to be sent to a server.  The server collects
these reports, and uses some kind of system (maybe rule-based, but
probably involving scores to estimate probability of spam or other
unwanted e-mail) to determine what is and is not spam, and with what
probability.  It then sends to each of its subscribers, every day or
so, a report indicating scores for users' messages.  (These should
be individualized.)

The mail filters then do one of four things to each piece of mail
rated, based on the scores:

a.   Pass the message through immediately.
b.   Discard the message immediately.
c.   Add the message to a list of low-priority messages, to be read
     when the user has some spare time.
d.   Send e-mail back to the sender, informing him of conditions
     under which the user is willing to accept this e-mail.  This
     might be things like requiring anonymous users to provide some
     minimal kind of identity, or telling senders ``I'll read your
     e-mail for one dollar in digicash,'' or ``I'll read your e-mail
     if you carry out this computationally expensive calculation, or
     some other thing.

The junk e-mailers can try various countermeasures to this.  The
most obvious are:

a.   Try to hit people who aren't using a good junk-mail filter.
b.   Try to make it against the law to use a junk-mail filter.
     (Perhaps this would be the case only for PSA spams?)
c.   Try to disguise their e-mail to make it not obviously junk
     e-mail, and simultaneously to alter each message to avoid
     detection by the servers, by making changes to each message,
     timestamp, and claimed sender ID.

I think (c) will be somewhat difficult for the junk e-mailers, if
the people who run the servers are reasonably clever.  The servers
should run indexes that find many identical or similar sentences,
paragraphs, etc, in messages sent to many people.  I think either
the junk e-mailers would give up on these filters immediately, or
there would be an endless arms race between advertisers and filter
servers.

There are some potential problems with this approach, though.  The
servers will be getting a lot of information about what e-mail is
coming to each of their users.  There will be serious privacy
concerns, especially if the filters work after decrypting public-key
encrypted messages.  (Note that if the user's public key is
reasonably long, PK encrypting the message will actually be pretty
hard for thousands or millions of messages at a time.  Also, there
will be various denial-of-service attacks, where I know Alice is
getting ready to send Bob some e-mail I don't want him to get, so I
intercept Alice's e-mail and forward it to 10,000 other people--thus
ensuring that it will be classified as spam.

Comments?

Note:  Please respond via e-mail as well as or instead of posting,
as I get CP-LITE instead of the whole list.

   --John Kelsey, jmkelsey@delphi.com / kelsey@counterpane.com
 PGP 2.6 fingerprint = 4FE2 F421 100F BB0A 03D1 FE06 A435 7E36

-----BEGIN PGP SIGNATURE-----
Version: 2.6.2

iQCVAwUBMj7vzUHx57Ag8goBAQGUbgQAsP62f0HDO4L0cs3DjCh9ppX3IgQUX8l6
W4JtH3WPfaHrzftD4UMGZ3D41kCjvGht/s62dPtq4lzDbqSpSB81oh4RVuyEw/kD
CZ4L0q2q6jFkTdnIp2mvP1WNlCTTpw2BBKY5U4tYCcthq8y30YmOGSqpKouK4l9S
gCV3Nd6C/Ig=
=Dent
-----END PGP SIGNATURE-----

JMKELSEY@delphi.com writes:
>                                                I just don't think
> I'd like Delphi to start filtering my e-mail without asking me for
> permission and instructions first.)

Likewise a number of AOL users don't like the idea of AOL deciding what
they can and can't read. (By the way one of the good things about AOL is
that they ignore all Usenet cancels.) AOL is promising to let its users
filter their incoming mail the way they want to by the end of September.

> Each user has a mail filter with a set of rules written either by or
> for that user.  The mail filter does one of four things with each
> piece of e-mail it receives:
>
> a.   It lets the e-mail through immediately.  (E-mail from friends,
> employers, employees, family members, etc. would probably be in this
> category.)

This should not be the default.

> b.   It discards the e-mail immediately.  (E-mail from people you
> really didn't like, and from people who have spammed you in the past
> would probably be in this category.)

Or may from mailing lists submitted by known idiots.

> c.   It puts the e-mail on hold in some storage area.

This should be the default for unknown senders.

> d.   Send e-mail back to the sender, informing him of conditions
>      under which the user is willing to accept this e-mail.  This
>      might be things like requiring anonymous users to provide some
>      minimal kind of identity, or telling senders ``I'll read your
>      e-mail for one dollar in digicash,'' or ``I'll read your e-mail
>      if you carry out this computationally expensive calculation, or
>      some other thing.
>
> For e-mail in the third category, some kind of summary report is
> sometimes generated, to be sent to a server.  The server collects
> these reports, and uses some kind of system (maybe rule-based, but
> probably involving scores to estimate probability of spam or other
> unwanted e-mail) to determine what is and is not spam, and with what
> probability.  It then sends to each of its subscribers, every day or
> so, a report indicating scores for users' messages.  (These should
> be individualized.)

One can simply choose to read the a) mail now and c) mail later.

Consider this scenario:

10:00 Eve sends junk e-mail to Alice and Bob.
10:05 Alice reads her urgent e-mail; leaves non-urgent e-mail for later.
10:30 Alice reads non-urgent e-mail, discovers junk mail from Eve.
10:31 Alice posts a warning to a Usenet newsgroup about junk e-mail from Eve,
 specifying a pattern than matches Eve's junk mail, and perhaps an address
 for postmaster complaints.
10:40 Bob starts reading his e-mail. Before he begins, his e-mail reading
 program fetches new e-mail notices from the usenet newsgroup, finds the
 ones from the issuers Bob trusts, checks their PGP signatures, and adds their
 patterns to its database. It then junks Eve's letter (discards it or bounces
 it to the postmaster, whatever Bob chooses)

Note that procmail can't do it - procmail would get Eve's junk mail at 10:00.
We want to delay the processing of the incoming queue to get the latest
available junk mail notices.

> The junk e-mailers can try various countermeasures to this.  The
> most obvious are:
>
> a.   Try to hit people who aren't using a good junk-mail filter.
> b.   Try to make it against the law to use a junk-mail filter.
>      (Perhaps this would be the case only for PSA spams?)

It's probably a bad idea for an ISP to impose such filters on users
without letting them "opt out", as AOL tried to do.

> c.   Try to disguise their e-mail to make it not obviously junk
>      e-mail, and simultaneously to alter each message to avoid
>      detection by the servers, by making changes to each message,
>      timestamp, and claimed sender ID.

That would an interesting technical challenge.

---

Dr.Dimitri Vulis KOTM
Brighton Beach Boardwalk BBS, Forest Hills, N.Y.: +1-718-261-2013, 14.4Kbps