[net.news] Why digests are bad

fair@ucbvax.ARPA (Erik E. Fair) (08/28/85)

Digested articles, of the form we see in fa.human-nets, mod.std.c,
mod.unix and fa.telecom are bad for the USENET, because they don't
allow the user to choose what he wants to read. Netnews is geared to
performing actions on single messages at a time. A digest breaks this,
because there is but one Message-ID per digest; one Subject per digest
(usually just the name of the digest and a volume number; of little use
for determining the actual content of the message); and one author (or
submitter) per digest, BUT *many* messages per digest!

This breaks our software in the following ways:

	Can't respond to individual items in a digest;
		only to the digest as a whole

	Don't automatically get mail replies sent to the right place

	Don't automatically get the right subject

	Don't get the ability to refer to previous items in the
		on the specific topic of one subject (i.e. no
		References: lines).

	Can't get a table of contents until you start reading the digest
		(and if you do get one at the top of the digest,
		you can't select which messages in the digest that you
		want to read).

Therefore, I propose that all moderators immediately stop digesting
items for submission to mod.* groups.

To clarify, I am not suggesting that we abandon moderation (e.g. the
selection of which articles to post by a designated person). I am
suggesting that the practice of clumping all submitted messages
together be stopped, because it defeats some important features of
netnews.

Comments?

	Erik E. Fair	ucbvax!fair	fair@ucbarpa.BERKELEY.EDU

whp@cbnap.UUCP (W. H. Pollock x4575 3S235) (08/29/85)

Diget articales CAN be broken up IF the news software is smart enough.
Readnews does this, rn ought to be able to breakup digests as they come in.
Perhaps putting the table of contents and final signiture block as the
base note, and individual articles as responses.

jmc@ptsfa.UUCP (Jerry Carlin) (08/29/85)

In article <10220@ucbvax.ARPA> fair@ucbvax.ARPA (Erik E. Fair) writes:
>Digested articles, of the form we see in fa.human-nets, mod.std.c,
>mod.unix and fa.telecom are bad for the USENET, because they don't
>allow the user to choose what he wants to read...
>
>Therefore, I propose that all moderators immediately stop digesting
>items for submission to mod.* groups.

AMEN! It is especially bad for people like me who are reading news
at 1200 baud.

I do think it is ok to group many items together if and only if they
are about the same topic.

Now if someone hacks rn to overcome Eric's objections and manages
to get all moderators (current and future) to agree to use whatever
method rn is setup to use that's another matter. Since I don't think
its very likely that this will happen, lets stop digesting.
-- 
voice= 415 823-2441
uucp={ihnp4,dual}!ptsfa!jmc

jww@sdcsvax.UUCP (Joel West) (08/29/85)

> Digested articles, of the form we see in fa.human-nets, mod.std.c,
> mod.unix and fa.telecom are bad for the USENET, because they don't
> allow the user to choose what he wants to read. 
	...
> Therefore, I propose that all moderators immediately stop digesting
> items for submission to mod.* groups.
> 

I agree.  What about digests that get forwarded pre-digested,
like fa.info-mac?  Do we leave as is, undigest it at cross
posting, or bite the bullet and add the appropriate digest-reading
logic to vnews and readnews?

If #3 is the best long-term course, then no administrative
changes are needed, only software. :-)

	Joel West	CACI, Inc. - Federal (c/o UC San Diego)
	{ucbvax,decvax,ihnp4}!sdcsvax!jww
	jww@SDCSVAX.ARPA

lauren@vortex.UUCP (Lauren Weinstein) (08/30/85)

I disagree.  The digests are all that make some groups tolerable.
Instead of plowing through message after message (most of which
will be useless at any given time) I can just scan the digest
heading and frequently throw ALL those messages out at once without
even hassling with them.  If by chance something worthwhile
appears in the heading, I go down to that particular message
and read it.  The percentage of messages that require replies
(for any given individual reading news) is incredibly less than the
number of "junk" articles that any particular person might
choose to ignore.  I consider digests to be one of the more useful
tools for cutting down on the amount of time required to wade
through the muck in search of the gems.  Digests aren't perfect,
but they sure help.

--Lauren--

heiby@cuae2.UUCP (Heiby) (08/30/85)

I was intending to post this in mod.unix exclusively, as it is the (belated)
summary of the responses that the readership of that digest sent to me when
I asked if mod.unix should remain in digest format.
--------
In response to my query, I received 12 messages in favor of keeping digest
format, 9 in favor of going to individual postings, and a couple that were
just general comments.  Based on this response, I have decided to keep
mod.unix as a digest.  Here is a summary of the comments I received.
Thanks to everyone who responded.  Ron.

	Summary of pro-digest messages:
----
I prefer the digest format because it is easier to skim; I can just look at
the subject list and decide (1) whether to read that issue and (2) how many
messages I can automatically skip on the way to the message I want.  It takes
me a lot longer to deal with individual messages, especially since the
advertised topics probably aren't accurate.  This is why I stopped reading
net.unix and net.unix-wizards.
----
Please keep the digest form.  I archive mod.unix, and digests
give me lower disk fragmentation.  I use rn, so am less bothered
by the low speed problem, even when I read news from home (like now).
----
Here's my vote: keep it a digest.  Easier to read, to maintain, and
to archive, and as you say, less traffic and disk resources are consumed.
----
Here's one vote for keeping mod.unix as a digest.  In addition to the
advantages you mentioned, digests also have the advantage of keeping
related articles near each other.
----
I much prefer the digest format.  All the points you give in its
favor are valid, and I would add another: the smaller amount of
traffic in the newsgroup is just easier to deal with (for me,
anyway).  I'd much rather read a digest less often than individual
articles more often.  Please keep mod.unix as a digest.


	Summary of anti-digest messages [with my comments]:
----
Now if rn were smart enought to work
with digests... (or if I were smart enought to figure out a nice way
of having rn work with digests...)
[Great news!  Version 4.3 of rn is smart enough to work with digests
(at least for some things).  There is the "gpattern" pager level request,
which searches forward for "pattern" in the current article.  There is
"G", which searches for the same "pattern" again.  There is "^G", which
"is a special version of the 'g' command that is for skipping articles
in a digest.  It is equivalent to setting '-g4' and then executing the
command 'g^Subject:'."]
----
Never mind the baud rate, it's much easier to read selectively if the
articles are posted separately.  Lumping them together just means that
the reader is likely to junk the whole thing rather than take the time
to skip through all the uninteresting stuff just to get to the only
article (s)he wants to read (invariably the last one!).
[The search capabilities of "rn" virtually eliminate this objection.]
----
Aren't the articles batched for transmission anyway, so you don't gain
anything by lumping them all together....?
[No.  Cycles are used to install and parse articles and disk space is
lost to headers and partial disk block use.]
----
I often like to read an
article or two during a compile or a lint. If the articles are bunched up,
I have to read part of a digest, then mark it unread if I can't finish
it right away. Then next time I read news I have to skip over the part
of the article I've already read...
[Good point, ghough rn's search can help as well as using the ! escape
to start other commands from within the news reader.  (or buy a DMD5620!)]
----
I might add to this, Ron, that although it seems like kind of a silly
thing to complain about, it makes a psychological difference; when
you're reading a huge posting, you just don't have that feeling that
you can "knock off a few articles" and go home. The feeling is more
that you *know* you will be interrupted in the middle of an article and
then will have to figure out where you were... I can't quite put my
finger on it but that's why I usually put off reading mod.unix 'till
"later when I have some time" which is usually never!
[I understand the feeling.  I do it myself sometimes.]
----
I too would prefer breaking it up into separate articles since
that makes it easier to respond to a particular article.
[Good point.  Maybe this should be a future enhancement to rn (if it
doesn't already).  (Reminder:  The Path header line isn't necessarily
a valid/optimal return address, especially if I am posting it from
my machine and the originator is not on my machine.)]
----
I would also prefer separate messages, for the reason of better
selectivity.  You perform the service of cutting the message traffic down
to just the "good" articles, but only I can cut it down to
subtopics-of-interest by examining the subject lines.
[Good news reading software can help this.]
----
I would prefer individual articles.  It gives you better possibilities to
read parts of the group, and come back later.  If you are a bit behind, you
might wish to read all articles on a certain topic at the same time, and
wait with the rest of the group.
[This is a good point.  I have no good answer.  The rn "m" or "M" command
can be used to mark the article (digest) as still unread, then you can
return to it and use the "g" command to find the part to which you want
to reply.  Another thing is that with a moderated newsgroup, duplicate
answers get trimmed down (at least in mod.unix) so that they appear only
if they provide new information or present it in a clearer/different
manner.]
-- 
Ron Heiby {NAC|ihnp4}!cuae2!heiby   Moderator: mod.newprod & mod.unix
AT&T-IS, /app/eng, Lisle, IL	(312) 810-6109
"No; my legs are written in a functional programming language." (J. McKie)
-- 
Ron Heiby {NAC|ihnp4}!cuae2!heiby   Moderator: mod.newprod & mod.unix
AT&T-IS, /app/eng, Lisle, IL	(312) 810-6109
"No; my legs are written in a functional programming language." (J. McKie)

lwall@sdcrdcf.UUCP (Larry Wall) (08/30/85)

In article <1067@sdcsvax.UUCP> jww@sdcsvax.UUCP (Joel West) writes:
>I agree.  What about digests that get forwarded pre-digested,
>like fa.info-mac?  Do we leave as is, undigest it at cross
>posting, or bite the bullet and add the appropriate digest-reading
>logic to vnews and readnews?

If you have lots of users, the best place to undigestify articles is in
inews/rnews, so that you only do the work once instead of 50 times, and
you do it at night instead of in the daytime, and you do it when nobody is
sitting at the terminal tapping their fingers and wondering if they should
go get some tea.

The only disadvantage that I can see is if you are tight for disk space,
you'd rather have 1 file than n files.  (Hmm, maybe inews needs to
transmogrify a digest into an archive, and then make n copies of it using
links.  Then anybody who wants to open an article has to check to see if
it's an archive, and zero in on that part of the archive containing the
article desired.  Yech.  How about just making undigestification optional?)

I've also been asked to make rn sort articles into chronological order, and
have refused for the same reason--I think it should be done by inews.

I guess I'm turning into an opinionated old man. 

Larry Wall
{allegra,burdvax,cbosgd,hplabs,ihnp4,sdcsvax}!sdcrdcf!lwall

gds@mit-eddie.UUCP (Greg Skinner) (08/30/85)

> From: fair@ucbvax.ARPA (Erik E. Fair)

> Digested articles, of the form we see in fa.human-nets, mod.std.c,
> mod.unix and fa.telecom are bad for the USENET, because they don't
> allow the user to choose what he wants to read. Netnews is geared to
> performing actions on single messages at a time. A digest breaks this,
> because there is but one Message-ID per digest; one Subject per digest
> (usually just the name of the digest and a volume number; of little use
> for determining the actual content of the message); and one author (or
> submitter) per digest, BUT *many* messages per digest!

I was under the impression that readnews breaks digests into single
replyable articles.  As long as the moderator puts a From: and Subject:
line into each digested article, I don't see how there can be any
problems.  In addition, they can put in Message-IDs also, although I
don't know how much good that will do for your general-purpose news
system, perhaps there are some fancier news systems out there that can
cross-reference on an arbitrary Message-ID?

> This breaks our software in the following ways:

>	Can't respond to individual items in a digest;
>		only to the digest as a whole

You can (with digesting in readnews -- is it in vnews yet?) -- see above.

>	Don't automatically get mail replies sent to the right place

If all moderators put in replyable paths including well-known hosts, or
at the least put in (or have the submitters include) a path to their
machine including well-known hosts, mail replies have as much a chance
of going to the right place as if they were posted singularly.  I've
replied to news articles and been rejected through news-only hosts -- I
was usually better off finding my own path to the machine (but those
stories are better left in net.mail).

>	Don't automatically get the right subject

Again, each digested article should have its own subject.

>	Don't get the ability to refer to previous items in the
>		on the specific topic of one subject (i.e. no
>		References: lines).

Maybe there could be Message-ID's in each digested article and
references in the moderated posting to those ID's?

>	Can't get a table of contents until you start reading the digest
>		(and if you do get one at the top of the digest,
>		you can't select which messages in the digest that you
>		want to read).

I thought that in readnews after the digest is broken you could select
which ones you wanted to read.

> Therefore, I propose that all moderators immediately stop digesting
> items for submission to mod.* groups.

I propose that we include my suggestions (or at least talk about them),
maybe some additional cross-referencing and timestamping of digested
articles can be made a part of the news software.
-- 
Do not meddle in the affairs of wizards,
for they are subtle and quick to anger.

Greg Skinner (gregbo)
{decvax!genrad, allegra, ihnp4}!mit-eddie!gds
gds@mit-eddie.mit.edu

gds@mit-eddie.UUCP (Greg Skinner) (08/30/85)

I did some playing around with digests, and I didn't realize that the
reply code was not in the digest code!  I could've sworn it was.  I
suppose that it would be relatively easy to put the reply code in the
digest code, assuming all other things that I suggested in my previous
message be done (inclusion of From: Subject: and other lines that
readnews would want for a news article).

Sorry for the mistake -- it's been awhile since I've looked at the news
sources, or played around with digests.  Just base everything in my
previous message on the inclusion of the reply code in the digest code
and we should be ok.
-- 
Do not meddle in the affairs of wizards,
for they are subtle and quick to anger.

Greg Skinner (gregbo)
{decvax!genrad, allegra, ihnp4}!mit-eddie!gds
gds@mit-eddie.mit.edu

smb@ulysses.UUCP (Steven Bellovin) (08/30/85)

I vote with Lauren -- I like digests.  This is especially the case when
the moderator takes the trouble to group related messages together, possibly
even composing single-topic issues of the digest.  As for load -- well,
there's a lot of per-article overhead; using digests cuts this down.

guido@boring.UUCP (09/02/85)

I use rn, I know about ^G and G, and I still don't like digests.  The main
problem is at 1200 baud (or lower).  Using rn's '=' command I can (and
routinely do) scan the headers of all articles on undigestified groups and
skip those I don't like (indeed I often skip everything), just as quick as
Lauren does with digests.

But when I use ^G or G in rn, it insists on displaying the whole page, while
normally at the beginning of an article it only shows a few header lines and
one or two contents lines.  When I want to go directly to the n'th message in
a digest, I have to use g and type a large fragment of its subject.

Together, these habits of rn make it more painful for me to read digests than
to read normal groups.  I understand moderators' attempts at reducing the net
traffic, but moderated groups are already low volume, high content groups, so
even undigestified they don't contribute much to net pollution.  Grouping
several articles with the same subject line together in a digest is fine for
me.

Someone is surely going to suggest to add some more quirks to rn's
undigestifying code.  Wrong.  Rn has very general capabilities to treat
articles and groups of articles, and it would need duplication of all these
features for digests.  And don't tell me to go back to readnews.

--
	Guido van Rossum, CWI, Amsterdam (guido@mcvax.UUCP)

jww@sdcsvax.UUCP (Joel West) (09/02/85)

> I use rn, I know about ^G and G, and I still don't like digests.  The main
> problem is at 1200 baud (or lower).  
> 		...
> 	Guido van Rossum, CWI, Amsterdam (guido@mcvax.UUCP)

I'd just like to second one point here.  When considering the
convenience/utility of certain features, be sure to consider those
of us using readnews/rn/vnews at 1200 baud.  We may not consitute
a majority, but we certainly are a significant minority.

Digests are a real pain in vnews (all I use) since it normally means
two whole pages in order to get the digest header, filler, and then
the individual article fillers.

Speaking of vnews, there was a hacked version of it (2.11, it called
itself) that I used in AT&T land.  It allowed you to have a menu
(somewhat like ucb/mail 'h') of 1-line article headers, skip and mark
individual articles, and bounce back between headers and articles.
Does anyone know why it's not part of 2.10.3?

	Joel West	CACI, Inc. - Federal (c/o UC San Diego)
	{ucbvax,decvax,ihnp4}!sdcsvax!jww
	jww@SDCSVAX.ARPA

PS: Things are only marginally better at 2400

mangoe@umcp-cs.UUCP (Charley Wingate) (09/04/85)

In article <764@vortex.UUCP> lauren@vortex.UUCP (Lauren Weinstein) writes:

>I disagree.  The digests are all that make some groups tolerable.
>Instead of plowing through message after message (most of which
>will be useless at any given time) I can just scan the digest
>heading and frequently throw ALL those messages out at once without
>even hassling with them.  If by chance something worthwhile
>appears in the heading, I go down to that particular message
>and read it.  The percentage of messages that require replies
>(for any given individual reading news) is incredibly less than the
>number of "junk" articles that any particular person might
>choose to ignore.  I consider digests to be one of the more useful
>tools for cutting down on the amount of time required to wade
>through the muck in search of the gems.  Digests aren't perfect,
>but they sure help.

BUt but but

If you're using rn, you're already getting all these things WITHOUT the use
of digests.  You can ask it to list the articles by subject, you can follow
out subject lines, you can move around to particular articles at will....

My feeling is that digests are a liability for rn users.  They defeat all
the nice tools that rn uses to organize articles for you.  It's also
apparent that readnews and (possibly) vnews people need digests to get these
aids.  So I guess I really don't have an answer for this.

Charley Wingate

peter@baylor.UUCP (Peter da Silva) (09/05/85)

> posting, or bite the bullet and add the appropriate digest-reading
> logic to vnews and readnews?

Readnews already has it. Are you volunteering to copy the logic to vnews?
-- 
	Peter (Made in Australia) da Silva
		UUCP: ...!shell!neuro1!{hyd-ptd,baylor,datafac}!peter
		MCI: PDASILVA; CIS: 70216,1076

fair@ucbvax.ARPA (Erik E. Fair) (09/14/85)

I get the distinct impression that everyone missed the point
(everyone, that is, except Larry Wall and Charlie Mangoe).

With the information contained in the header of a news article,
your wonderful whizzy user-interface can construct a digest of all
the articles posted on September 10th, 1985 if you want.

It is wrong for the user interfaces to have undigestifying code in
them, because they should not have to deal with two kinds of article
structure. And because there is more useful information in the header
of single netnews articles than there is in the header of a generic
digest article (no message-id, or references, ...), there are more
things you can do with single netnews articles.

THAT is why digests are bad.

	1) They force us to have crufty code to deal with them in
		the user-interfaces (which from a design point of
		view is wrong)
	
	2) They break existing and potential functionality otherwise
		provided by a netnews article header, by not having
		the requisite information.

Larry Wall is also correct in stating that sorting of netnews articles
is the province of rnews, and therefore refusing to provide such
capability in rn (among other things, that would slow it down).

	Erik E. Fair	ucbvax!fair	fair@ucbarpa.BERKELEY.EDU

peter@graffiti.UUCP (Peter da Silva) (09/16/85)

Well I have never in my life read news at anything above 1200 baud, and I
prefer digests. What I generally like about digests is if I'm short on time
I can print them & read them at my leisure, something I can't do with regular
messages.

bart@reed.UUCP (Bart Massey) (09/17/85)

> > posting, or bite the bullet and add the appropriate digest-reading
> > logic to vnews and readnews?
> 
> Readnews already has it. Are you volunteering to copy the logic to vnews?

	Our version of the vnews sources already has it also -- but it's
'#ifdef'ed out because it's broken.  It would thus be a matter of fixing
it rather than writing it -- which may not be easier.  I think the fix
is non-trivial, anyway, based on my trivial inspection of the sources,

					Bart

mike@peregrine.UUCP (Mike Wexler) (09/25/85)

The people that like digests seem to like them because they are easily
printed.  Why couldn't somebody that likes printing out articles create
a program that would take a news group, a set of related articles(same
subject or parent chaining) and create one article suitable for printing.
This way people who are ready the articles on line can take advantage of
there news software without have them hacked apart to understand digests.


-- 
Mike(always a dreamer) Wexler
15530 Rockfield, Building C
Irvine, Ca 92718
(714)855-3923
(trwrb|scgvaxd)!felix!peregrine!mike

pwd@pid.UUCP (Philip W. Dalrymple) (10/02/85)

In article <192@peregrine.UUCP> mike@peregrine.UUCP (Mike Wexler) writes:
>
>The people that like digests seem to like them because they are easily
>printed.  Why couldn't somebody that likes printing out articles create
>a program that would take a news group, a set of related articles(same
>subject or parent chaining) and create one article suitable for printing.
>This way people who are ready the articles on line can take advantage of
>there news software without have them hacked apart to understand digests.

I do this now by saving the articles (with rn s command) then just printing
the file. It works just fine.

-- 
Philip Dalrymple
akgua!pid!pwd
404/429-8266 (voice)

nazgul@apollo.uucp (Kee Hinckley) (10/11/85)

...
>In article <192@peregrine.UUCP> mike@peregrine.UUCP (Mike Wexler) writes:
>>
>>The people that like digests seem to like them because they are easily
>>printed.  Why couldn't somebody that likes printing out articles create
>>a program that would take a news group, a set of related articles(same
>>subject or parent chaining) and create one article suitable for printing.
>>This way people who are ready the articles on line can take advantage of
>>there news software without have them hacked apart to understand digests.
>
>I do this now by saving the articles (with rn s command) then just printing
>the file. It works just fine.
>

So long as any news program lets you batch stuff (like readnews) doing your
own 'digests' is no problem.  I have a bizzare hacked together collection
of scripts that runs every night and collects news groups, nroffs them,
creates a table of contents and permuted index and dumps them to the printer.
Then I can read them at my leisure.  (This also leads to bizzare statistics
like:  net.sf-lovers once spent 60 pages discussing the tv show 'V'!!!!?)

                                                -nazgul

                                    ...decvax!wanginst!apollo!nazgul


        The Hydrogen Dog and the Cobalt Cat,
        side by side in the armory sat.
        Nobody thought of fusion or fission,
        everyone spoke of their peacetime mission.
        Till somebody came and opened the door
        and they they were in in a neutron fog;
        the Codrogen Cat and the Hybalt Dog.
        They mushroomed up with a terrible roar,
        and nobody, never, was there no more.
        
                            "The Space Childs Mother Goose"