Is cypherpunks archived somewhere?

Is the cypherpunks list being automatically archived where it can be
retrieved by anonymous FTP?

Yes, there *are* occasional nuggets on this list. Why, there's even
the occasional practical item about cryptography buried in the massive
rants to the libertarian choir. But I'd hate to miss stuff like the
recent note on RC4. That's why I haven't unsubscribed yet.

But something has to give. The list is just too high-volume, and the
signal-to-noise ratio is now below -1.6 dB. It's just not worth it.
It's a major reason why I always seem to run as much as a week behind
on my mail.

If there were an automatic FTP archive of the list, then I could
unsubscribe while keeping the option of pulling over and grepping
through an archived batch every so often just to see if anybody is
actually doing anything.

Phil

-----BEGIN PGP SIGNED MESSAGE-----

Phil Karn writes:
 > But something has to give. The list is just too high-volume, and the
 > signal-to-noise ratio is now below -1.6 dB. It's just not worth it.
 > It's a major reason why I always seem to run as much as a week behind
 > on my mail.

     In the past I've seen mention of using extropian-list-style
software to enable recipient control of which threads to receive.  Is
this a possibility?

     (And, yes, you may consider this an offer of some time to help
set it up and/or test it, if I may be of use.)

Regards,

Patrick May
pjm@gasco.com

-----BEGIN PGP SIGNATURE-----
Version: 2.6

iQCUAwUBLncfuRByYwhWPvz1AQEHhQP3U93lCxz58Hg0CEopEZ7utoOsf0IUt/25
d0eZ2EA7Lc2+WzOq6eAoFbmv+vuJdEx57w5M+18Pv8uw5YnWEWGQ4C6V2arhVwpo
XPca5ZoMiCiyyTYlq6OIxQL0472PNm5xVtiN0b+WSmbGpsLFBkE+mFktJduqGAa/
Du09kxL7mw==
=5EZY
-----END PGP SIGNATURE-----

The filtering/reptuations/volume issue has come up again.

And a couple of people are planning "reputation servers." I applaud
them for the effort, but I don't see them as needed for our list,
right now. That is, I don't plan to use them.

* what most people need are tools to filter out messages they don't
want to read. As we don't yet have quasi-intelligent agents that can
do this for us, this generally involves:

- filtering based on thread
- filtering based on author

* having these tools is more important to me than having a database of
what other people think about other people (reputation data bases).

There are several approaches:

* Unix kill-files, for those with the shells that support them

* Eudora-type filtering (ironically, Eudora is sold by Qualcomm, Phil
Karn's company), which allows various kinds of sorting.

* Extropians-style filtering, now being developed as a commercial
product by Harry Shapiro and Ray Cromwell. (To be clear about things,
Hugh Daniel, Eric Hughes, etc., expressed their willingness to install
the Extropians-style software shortly after it became available,
almost 2 years ago. Various delays ensued, then the offer by the
authors was put in limbo, then the commercialization phase ensued.)

* Anyone can operate a refector for the list, as per several
statements on this. Hal Finney, for example, offered (offers?) an
encrypted-only version. My point: someone could set up a filtering
service, a digester, whatever, and others could subscribe.

(Yes, Robert Hayden did this for a few weeks. While it may not have
been his "fault" that it went down, it shows the generally flaky and
catch-as-catch-can nature of so many part-time, hobbyist systems. Like
the remailers that go down when the laptop running it gets taken to
Spain for the summer :-}. A "for profit" service, at some quarterly
fee and with a contractual relationship to continue service, is a
better long-term approach.)

I sympathize with the concerns of Phil Karn and others, but let me
give a warning about this. The Extropians list, when I was on it, had
about the same volume at its peak that we now have, about 50-100
messages a day. Much debate about S/N ensued, much talk about charging
a fee for posting, about setting up quotas, about about official
reputation markets. At least 20% of all list traffic in some weeks was
devoted to kvetching about this problem.

A "reputation market" called the Hawthorne Exchange was set up, as
discussed here by Hal Finney (and also by me in my FAQ..grep for
Hawthorne or HEx). Even more jawboning went on.

And then of course there what the new list software. This allowed
folks to exclude authors, threads, etc., at the _point of
distribution_.

Is this a good idea.? Well, if one excluded 10% of the traffic, then
it would "save" having to receive 5-10 messages a day. Big deal.

I used the ExI software, and found it an interesting experiment, but I
can't say it save me any real effort. The effort of sending the
filtering message to the list site, hassling with the formats, etc.,
clearly outweighed the tiny effort it would have taken to manually
press "d" to delete the messages when then appeared.

I also found it useful to at least spend the 2-5 seconds to see what
was being talked about before pressing "D."

(A side issue: Whatever seconds were saved by the distribution-point
filtering (and I haven't mentioned the CPU time required...an issue
for us to consider with 700 list members) on the Extropians list were
often negated for the others by people asking "What are you talking
about?" or "Could someone send me Joe's posting on foo--I had him in
my ::exclude file.")

In other words, I find just being real fast on the "D" key is my best
way to cope with list volume.

Your mileage may vary, but I doubt that the Extropians-style software
is going to help much...I used it, and my experiences are what I just
described. 

Filtering is the wave of the future.

Paul Baclace, sometimes on this list, was working on filter agents for
Usenet that could learn preferences. And I've seen such things with
WAIS.

--Tim May


-- 
..........................................................................
Timothy C. May         | Crypto Anarchy: encryption, digital money,  
tcmay@netcom.com       | anonymous networks, digital pseudonyms, zero
408-688-5409           | knowledge, reputations, information markets, 
W.A.S.T.E.: Aptos, CA  | black markets, collapse of governments.
Higher Power: 2^859433 | Public Key: PGP and MailSafe available.
"National borders are just speed bumps on the information superhighway."

>I sympathize with the concerns of Phil Karn and others, but let me
>give a warning about this. The Extropians list, when I was on it, had
>about the same volume at its peak that we now have, about 50-100
>messages a day. Much debate about S/N ensued, much talk about charging
>a fee for posting, about setting up quotas, about about official
>reputation markets. At least 20% of all list traffic in some weeks was
>devoted to kvetching about this problem.

This is not what I asked for. I already have a pretty good filter:
it's called "grep". Plus a few friends have volunteered to forward the
occasional worthwhile article to me by private email. All I wanted was
an alternate transmission mechanism - complete archives on demand by
anonymous FTP rather than automatically by email. Is this so
difficult?

Even if I used Eudora regularly (which I don't), the problem wouldn't
get much better. In some ways it would get even worse. Eudora is very
nice for those who travel frequently: you can download mail in a big
batch whenever you can find a phone, read and edit it offline (e.g.,
during a flight), and then transmit your responses when you find
another phone.

The problem, of course, is the severely limited bandwidth of the phone
link. Sometimes I only have 10 minutes between flights. I simply don't
want to waste the time downloading the day's several megabytes of
cypherpunk flamage. Filtering it out after I've received it doesn't
help me.

Some have suggested netnews relays. This is not sufficient either,
given that most sites I know keep news for only a few days
to keep their disk space requirements within reason.

Is it so hard to add an entry to the cypherpunks list on toad.com so
that every message can be appended to a file in the anonymous FTP
area? It's not like it has never been done before...but perhaps that's
what makes it uninteresting to this group.

Phil

Phil Karn <karn@unix.ka9q.ampr.org> writes:

>Is it so hard to add an entry to the cypherpunks list on toad.com so
>that every message can be appended to a file in the anonymous FTP
>area? It's not like it has never been done before...but perhaps that's
>what makes it uninteresting to this group.

Script started on Thu Sep 15 14:11:52 1994
jobe% telnet toad.com 25
Trying 140.174.2.1 ...
Connected to toad.com.
Escape character is '^]'.
HELO
220 toad.com Sendmail 4.1/Gnu-smail ready at Thu, 15 Sep 94 14:12:08 PDT
250 toad.com Hello  (jobe.shell.portal.com), pleased to meet you
EXPN cypherpunks-outgoing
250-<hughes@ah.com>
[hundreds of names elided]
250-</proj/gnu/cypherpunks>
250-</u/hughes/mail/maillists/cypherpunks.archive>
250 </n/u/gnu/misc/crypt.cypherpunks.log>
QUIT
221 toad.com closing connection
Connection closed by foreign host.
jobe% exit
jobe% 
script done on Thu Sep 15 14:13:16 1994

This suggests that there are three possible files which are already
archiving the list.  How frequently they are deleted is another matter.
The list volume is so high that the disk space to hold much of an
archive becomes a bit expensive.  Still, if one of thse could be made
accessible to anon ftp it might be worthwhile (if toad allows anon ftp).

Hal

Still, if one of thse could be made
   accessible to anon ftp it might be worthwhile (if toad allows anon ftp).

Toad.com does not run an ftp daemon, and I can't install one myself.

Eric

In article <199409151957.MAA00506@unix.ka9q.ampr.org>,
Phil Karn <karn@unix.ka9q.ampr.org> wrote:
>Some have suggested netnews relays. This is not sufficient either,
>given that most sites I know keep news for only a few days
>to keep their disk space requirements within reason.

Just FYI: hks.lists.cypherpunks on bb.com's NNTP server goes back
 to July 16th: We're not expiring it, and don't plan to in the near
 future.  I could make it available for ftp, or even sup, if at least
 several folks were interested.
-- 
L. Todd Masco  | "A man would simply have to be as mad as a hatter, to try and
cactus@bb.com  |  change the world with a plastic platter." - Todd Rundgren

In article <199409152120.OAA27178@jobe.shell.portal.com>,
Hal <hfinney@shell.portal.com> wrote:
>The list volume is so high that the disk space to hold much of an
>archive becomes a bit expensive.  Still, if one of thse could be made
>accessible to anon ftp it might be worthwhile (if toad allows anon ftp).

Let's not get too silly here: disk space is cheap.  The aforementioned
 bb.com cypherpunks newsgroup takes up 7 megs right now, since July 16,
 and that's not even compressed (one of these days, I'll take care of
 that). That's $5 at current disk prices.  I'm certainly willing to
 shell out $2.50/month to keep all of cypherpunks around: I frequently
 grep for references.

No, it's bandwidth of a gazillion cypherpunks getting the material
 that worries me...  but we'll live with it.
-- 
L. Todd Masco  | "A man would simply have to be as mad as a hatter, to try and
cactus@bb.com  |  change the world with a plastic platter." - Todd Rundgren

-----BEGIN PGP SIGNED MESSAGE-----

tcmay@netcom.com (Timothy C. May) once said:

TC> The filtering/reptuations/volume issue has come up again.                   

    [massive deletia]

TC> (Yes, Robert Hayden did this for a few weeks. While it may not have         
TC> been his "fault" that it went down, it shows the generally flaky and        
TC> catch-as-catch-can nature of so many part-time, hobbyist systems. Like      
TC> the remailers that go down when the laptop running it gets taken to         
TC> Spain for the summer :-}. A "for profit" service, at some quarterly         

Well, the remailer@desert.xs4all.nl is BACK ! Note the CHANGE OF
ADDRESS ! I enjoyed my time in Spain, working as a DJ and
PR-person. I used the laptop only once, to look up an address....

There's also a new version of the remailer software for Waffle 1.65.
The current version is now 2.02, and it supports now "Anon-To",
"Anon-Alternate-ID" now (only in 'penet' mode), and it now uses
the SPAWNO routines by Ralf Brown to minimize memory use while
running other programs (e.g. PGP).

    [more deletia]

greetings,

    Patrick


== kafka@desert.xs4all.nl: Cryptoanarchy, MDMA, Tekkkno, SL-1200  ==
    ^^^^^^^^^^^^^^^^^ NEW EMAIL ADDRESS !!!!!!!!!!!!!!!!
    1024/322D0305 : F6 F1 96 54 97 F3 FF 54  49 EF BF 34 3E 3F 0C 60
     512/7994F705 : 79 8E 00 DB D5 A0 8D 62  02 C0 70 EF 76 A4 21 D0


-----BEGIN PGP SIGNATURE-----
Version: 2.6

iQBVAwUBLng34pRymF15lPcFAQGhDAH9FJZ4/I9hQi0nflNHIHCFi5ai8y9czTpq
Jm8XX0Hvv3ToULQ2+dvfmBbTPAEj3TFZnBQG27t1FuO/Q/s8sFZ64Q==
=AVDV
-----END PGP SIGNATURE-----

> * Extropians-style filtering, now being developed as a commercial
> product by Harry Shapiro and Ray Cromwell. (To be clear about things,
> Hugh Daniel, Eric Hughes, etc., expressed their willingness to install
> the Extropians-style software shortly after it became available,
> almost 2 years ago. Various delays ensued, then the offer by the
> authors was put in limbo, then the commercialization phase ensued.)

   Just to clear this up, this freeware code was never put into
limbo. I believe Harry talked to Hugh several times in e-mail
but Hugh didn't have the time to put up with the installation
(hint: it's harder than majordomo to install since they are
tons of configuration options.  You know, like the list statistics,
what filtering agents to use, digest format, incoming filters,
accounting database, etc)

   If anyone is willing to provide a machine, I will gladly
give and/or install my mailing list software, and then subscribe it
to cypherpunks. (during the development phase, this is exactly
how me and Harry tested the list. We subscribed to several mailing
lists and had the list software do the filtering)

   This is more optimal anyway. If the vast majority of people don't
want filtering, there is no need to waste cpu cycles by running
it as the main cypherpunks list. 

> And then of course there what the new list software. This allowed
> folks to exclude authors, threads, etc., at the _point of
> distribution_.

> Is this a good idea.? Well, if one excluded 10% of the traffic, then
> it would "save" having to receive 5-10 messages a day. Big deal.

> I used the ExI software, and found it an interesting experiment, but I
> can't say it save me any real effort. The effort of sending the
> filtering message to the list site, hassling with the formats, etc.,
> clearly outweighed the tiny effort it would have taken to manually
> press "d" to delete the messages when then appeared.
 
    Yes, and this is a recognized problem, and one of the things
we are going to eliminate in the commercial version. Basically,
the "send a msg to a mail server" form of command processing is
too painful to use except infrequently (such as ::resend) The
technique of using "in-band" commands within a post
was my first attempt to reduce the pain of sending commands
(by allowing you to postpone any commands, and then include
then in one of your posts later)

One of the ways to alleviate "transaction cost" of list commands is to
use client side scripts.  For instance, for list administration, I
wrote Harry a "hot key" perl script which takes over elm's Print
function. When Harry needs to do something, he hits 'p', and then
chooses an option. (for instance, to add a user, he types
'pa'. I could easily create a similar thing for anyone using
elm who wanted to exclude based on a single key-press)

> I also found it useful to at least spend the 2-5 seconds to see what
> was being talked about before pressing "D."

   The real use of the Extropians software comes in "exclude all" mode,
resend thread, and digest. Most of the people I see using the
list filtering (from the logs), do something of the following:
1) exclude all
2) read the 'filterlist' every 12 hours. If something looks interesting,
do ::resend thread, and get a digest containing that thread only.
3) include specific threads and authors.

   The only thing I ever use the filtering system for is to exclude
individual users, or annoying threads. (d for everything else, like
you)

> Your mileage may vary, but I doubt that the Extropians-style software
> is going to help much...I used it, and my experiences are what I just
> described. 

   I think it is superior to majordomo and listserv for other reasons
(remember, the filtering stuff is merely a "plug in" agent which could
be replaced with any filtering or reputation system) For instance,
the reputation system writers could use it as a platform to write
a reputation based filtering system if they wanted to.

> Filtering is the wave of the future.

  Definately. I fear there will be no good solution though until
we get atleast some partial natural language understanding.

-Ray

-----BEGIN PGP SIGNED MESSAGE-----

Timothy C. May writes:
 > 
 > I used the ExI software, and found it an interesting experiment, but I
 > can't say it save me any real effort. The effort of sending the
 > filtering message to the list site, hassling with the formats, etc.,
 > clearly outweighed the tiny effort it would have taken to manually
 > press "d" to delete the messages when then appeared.

     Like the use of PGP, this may be due to the different tools in
use.  I read email using the vm package for emacs.  While I was on the
extropians list I found it very convenient to reply to the first
message of a topic I was not interested in, modify the address, and
enter the simple ::exclude message.  Other tools and environments
could make this more difficult.

 > (A side issue: Whatever seconds were saved by the distribution-point
 > filtering (and I haven't mentioned the CPU time required...an issue
 > for us to consider with 700 list members) on the Extropians list were
 > often negated for the others by people asking "What are you talking
 > about?" or "Could someone send me Joe's posting on foo--I had him in
 > my ::exclude file.")

     Good point.


- ------------------------------------------------------------------------
                                A contract programmer is always intense.
Patrick May
pjm@gasco.com (public key available from servers)


-----BEGIN PGP SIGNATURE-----
Version: 2.6

iQCVAwUBLndwmhByYwhWPvz1AQHWbQP+MSM9YF0+OE1cPtDDSOUJhPyKNR3u7Zsi
9YBMH5TBTgh0TZCe6vs7EdSMXugRnvz0zvemqb0QrhVraTMbG70ecGnVZA5NP8pW
NzRn1Id3jktYgXgzCvW8DDIx0YSL5apYlK2Zm43qAMpQZRWhIHeZNVRIrTtmUbJG
PcagmiK2EfA=
=mms6
-----END PGP SIGNATURE-----

One of the things that might be helpful with regards to filtering would be
some kind of a user-friendly interface that will allow easy editing and
manipulation of the elm filter or procmail rules.  (For example, the Tin
newsreader has a good entry screen for killfiles based on subject or
author.) In addition, I remember way back when when I was using NN as a
newsreader, there was a way to set up killfiles with a certain number of
days before they would timeout and be removed from the killfile. 

If a program existed that would allow similiar manipulation of mail
killfiles, that would be great.  (regretably, I am a dreadful programmer
and really am not sure how to design or write the program). 

As for the digested version of this list, I have received good word and 
hopefully my machine will be back online in the next couple days.

____        Robert A. Hayden       <=> hayden@krypton.mankato.msus.edu
\  /__          -=-=-=-=-          <=>          -=-=-=-=-
 \/  /  Finger for Geek Code Info  <=> I do not necessarily speak for the
   \/   Finger for PGP Public Key  <=> City of Mankato or anyone else
-=-=-=-=-=-=-=-
(GEEK CODE 2.1) GJ/CM d- H-- s-:++>s-:+ g+ p? au+ a- w++ v* C++(++++) UL++++$ 
		P+>++ L++$ 3- E---- N+++ K+++ W M+ V-- -po+(---)>$ Y++ t+ 5+++
		j R+++$ G- tv+ b+ D+ B--- e+>++(*) u** h* f r-->+++ !n y++**

You wrote:

| One of the things that might be helpful with regards to filtering would be
| some kind of a user-friendly interface that will allow easy editing and
| manipulation of the elm filter or procmail rules.  (For example, the Tin
| newsreader has a good entry screen for killfiles based on subject or
| author.) In addition, I remember way back when when I was using NN as a
| newsreader, there was a way to set up killfiles with a certain number of
| days before they would timeout and be removed from the killfile. 
| 
| If a program existed that would allow similiar manipulation of mail
| killfiles, that would be great.  (regretably, I am a dreadful programmer
| and really am not sure how to design or write the program). 

	The rep. credit system that I sketched out a few days ago would
alliviate the need to edit your procmail rules by hand for those mail
message you choose to filter.  The way I had pictured setting it up
would have a procmail rule which would query a reputation database
(stored in the users account.)  The query would return a number, which
procmail could then act on.

	No timing features at the user level, but I've considered
putting in a decaying value for credit, to prevent entries from living
forever.  I doubt this would be in early versions.

	Lastly, I'm getting around to sketching out data structures,
the only problem I have to address in theory is how to prevent the
system from becoming a spam factory; deluging people who don't use the
system with piles of messages that they don't want.  Several inelegant
server based solutions appear (they often do), but I'm hoping to
design something more elegant.

Adam