[news.sysadmin] the USENET problems

page@ulowell.cs.ulowell.edu (Bob Page) (07/10/87)

[This article is over 100 lines and contains no flames.  It even
contains constructive proposals.  Think about them, please.]

Whenever someone gets mad at "the backbone" and flame about it, there
is always someone who posts this "response," or one like it:

	If you don't like the current (foo), start your own.

While this might be satisfying for the poster, it means nothing.
Sure it would solve the problem, but that's not the way to solve it.
Isn't it better to listen to the complaints-hidden-in-flames and
respond constructively?  Fragmenting USENET into dozens of international
subnets is a waste of resources.

I certainly don't have the answer to either side's comments, but
am interested in a solution.  Attacking/defending the backbone does
not help.

So what problems do we have?

1. The namespace (newsgroup names) are effectively controlled by
   Gene Spafford's checkgroups message.  It synchronizes everyone
   to the same set of newsgroups, and only the admins who really
   understand the net will bother to keep groups that don't appear
   in the checkgroups message.  Then again, those admins have to
   rely on their upstream sites to do the same.

2. Moderation doesn't work correctly because it introduces delays
   and is unreliable, since it uses point-to-point mail to get to
   the carbon-based moderator, who may or may not be there
   (witness mod.sources).

3. There's too much traffic (almost 2MB/day now) and much of it is
   noise.

Those are probably the three major problems; the rest are subsets
and offshoots of these.

Many would say that (1) isn't a problem at all.  A central authority
is great, it keeps the world in sync.  Since it is made up of people
who truly care about USENET, and is a somewhat large body, we can
assume that dictums from the backbone are generally good for the net.
Sure USENET is an anarchy, but most (>99%) of the site admins will
go along with the backbone, for various reasons (no need to list them).

reid@decwrl.UUCP (Brian Reid) points out that:
>... nntp and PC Pursuit have made the whole concept of the backbone quite
>obsolete.

Which really reduces the 'backbone' to a group of site admins who want
to steer USENET a certain way.  It is not a coincidence that many
of the people in this group are also people who work hard outside
the backbone for USENET (like Rick maintaining the news 2.11 software,
Henry working on C news, Mark coordinating the UUCP Project, Gene
publishing documentation, Brian collecting arbitron data, etc).  Of
course there are others in the 'backbone' who do little else for
the UUCP/UNIX/USENET community, but that doesn't mean they don't care
about the community.  Some admins on the backbone, I dare say, are
there soley because their employers are generous enough to put up
a lot of money to maintain UUCP connections.  Their main interest is
in keeping the news flow below a certain level.  Still nothing wrong
with that; they're indirectly concerned with USENET in that they
want more info and less noise.

It is possible that there are people on the backbone who are there
soley to feel like they have control, power, influence, etc.  Maybe
*all* of the admins feel that to some degree.  I suppose it's part
of the society we live in; but I digress, and point out that they are
probably not any kind of major influence on USENET.

Now that I hope I have defined the backbone in non-emotional terms,
may I suggest we drop the name and change it to something more realistic?
How about USENET Steering Committee?  Also, how about opening the
group to a (somewhat) larger body?  There are people (admins, non-admins)
who care about making USENET better who are not on the 'backbone' list.


The answer to (2) is more complex.  I don't think anyone is truly
satisfied with the way moderation works now.  Most USENET readers
and posters probably agree that moderation in general is a good idea,
but the current implementation makes it difficult for posters to get
information to readers.

The answer is to come up with a different way to do moderation.  I don't
have solid proposals on what that "different way" should be.  I'd like
to see a new newsgroup that deals with this question.  When a good
proposal can be hammered out, implementation can begin.  B news 2.12 maybe.


The third problem is due to the increase in the number of UNIX systems
in use now.  Since we can't get rid of these new UNIX sites, we have
to find some other ways to decrease the 'noise' traffic.  Some are
software based, like the 50% included text limit or the 4-line signature
limit.  Although controversial (and without test data to demonstrate
any effectiveness), they are a step towards limiting unneeded text.
Another solution is better user education and admin education.

Probably the best solution is to break the bulk of USENET into
regional networks, with regional group 'forwarders' who send the best
(mostly signal, little noise) postings to the larger nets (country
net, world net, etc).  An ideal implementation would have stratified
nets, with some limit on the level (like I can post to the ULowell
net, Massachusetts net and New England net, but not the USA net,
the North American net, the Earth net or anything higher).  Appropriate
postings that I make to the (foo) group in New England might get sent
to the USA net for countrywide distribution.

In some ways it's like the company newsletter, where an article gets
picked up by the town weekly paper and the regional daily paper.  Maybe
USA Today picks it up for national distribution.  Maybe UPI and AP pick
it up for distribution to other countries and networks, etc.  In other
ways it's not like the company newsletter, etc, but you get the idea.
As you go up in levels, you get reduced noise in newsgroups.  You'll
still have netwide discussions (like the future of news, unix, etc),
but when you come right down to it, there's no need for netwide
discussions on (foo) when most of it is noise.

Opponents who say "But what if somebody in Texas says something
that I would otherwise be interested in?  Leave it the way it is!"
aren't being constructive in reducing noise (or finding some way
to make this regional net scheme work).  My quick reply is "what
if some newspaper in Texas says something that I would otherwise
be interested in?"  You can't read everything, most of it would
be noise, right?  Well, look around you.  Welcome to USENET.

I'd like to see a new newsgroup dedicated to discussing/solving the
traffic issue as well as one for the moderation problem.

Reasonable comments welcomed.  Flames OK too (email, don't post) as
long as there's some light somewhere amongst the heat.

..Bob
-- 
Bob Page, U of Lowell CS Dept.   page@ulowell.{uucp,edu,csnet} 

cas@mtuxo.UUCP (07/10/87)

It seems like we might end up trying something like the thing Fidonet did.  They
are a lot younger and started (?) with a master governing group, the
International FidoNet Association (IFNA).  I remember there was talk a coupla
years ago on a Usenet corporation, but it didn't get far.  Maybe it's time to
rethink! 

srt@CS.UCLA.EDU (07/10/87)

There are different kinds of moderation needed.  One type is source group
moderation.  The moderator is primarily responsible for seeing that the
postings are actually source code and don't contain any obvious Trojan
Horses.  A second type of moderation is used in opinion groups, where
moderation is used to winnow out uninformed opinions and repetitive comments.

I've suggested before that the first type of moderation can be handled by
forcing posting to source groups through a local (i.e., site) moderator,
normally the new administrator.  This is an additional burden on the
news admin, but since most sites rarely post more than one piece of source
a month, it would hardly be overwhelming.

This is even implemented quite easily; you change the moderator's address
to be the address of the local moderator.
 
    Scott R. Turner
    UCLA Computer Science     "I don't need your plastic hassles."
    Domain: srt@ucla.cs.edu
    UUCP:  ...!{cepu,ihnp4,trwspp,ucbvax}!ucla-cs!srt

jesup@mizar.steinmetz (Randell Jesup) (07/11/87)

[page@ulowell.cs.ulowell.edu said one of the current problems with the net
is the lack of reliability and promptness in moderated groups.]

	This is definitely a problem.  Mailers can be unreliable, moderators
go on vacation, work at jobs, have personal lives.  Often (in the more
popular groups), I suspect they get overloaded.
		Proposed solution/improvement for comment:
	How about when you post to a moderated group, it checks to see who
the moderator is (as it does now).  If there is more than one(!), put up 
something like:
		Mail to which Moderator?
			(1) page@ulowell.cs.ulowell.edu (shortest path)
			(2) jesup@ge-crd.arpa
			(3) anyman@anyunix.uucp
	It could have different moderators for different parts of the net
(hard), or could try to estimate which is the easiest to reach (not easy),
or not bother, in deciding the ordering.  That way, when moderators go on
vacation, the users could route to the others, etc.  The moderators could
even send out control messages saying they'll be gone for a period, and not
have their name come up.  
	This would also allow each moderator (if there are more than one) to
to handle a reasonable flow, and improve the reliability in cases where 
moderators seem to 'disappear' for periods of time.
	Another variant would be to have some sort of 'regional' moderators,
probably not very easy given the current net topography.

	If the backbone is so enamored of moderated groups, I assume they're
interested in making moderation work well enough to be more acceptable for
the rest of the net.

	Opinions?

		Randell Jesup
		jesup@ge-crd.arpa
		jesup@steinmetz.uucp

farren@hoptoad.uucp (Mike Farren) (07/11/87)

In article <193@mtuxo.UUCP> cas@mtuxo.UUCP (C.STEVENS) writes:
>
>It seems like we might end up trying something like the thing Fidonet did.  They
>are a lot younger and started (?) with a master governing group, the
>International FidoNet Association (IFNA).  I remember there was talk a coupla
>years ago on a Usenet corporation, but it didn't get far.  Maybe it's time to
>rethink! 

Fidonet didn't start with a governing body, but as an anarchical group of
BBS systems which happened to be able to communicate with one another.
IFNA developed much later on.  It is worth noting that Fidonet as a
whole does NOT agree that IFNA is a good thing, and there are certain
arguments around that it was started as a power ploy by certain Fido
sysops who wanted to have the power to determine how Fidonet was
"supposed to be run".  I have my own opinions on the matter (which
probably can be deduced by careful reading of the above) but, more to
the point, isn't this EXACTLY what would happen if a Usenet governing
body were formed?  Given that the bitching goes on for weeks if Spaf
just makes suggestions, let alone wields real decision making power, I
think that we should all think about the repercussions of establishing
a *real* Usenet cabal very carefully indeed.  I don't know if it would
be a good thing or not, but I do know that the decision would cause at
the very least an uproar like we've not seen yet (renaming not
withstanding).


-- 
----------------
                 "... if the church put in half the time on covetousness
Mike Farren      that it does on lust, this would be a better world ..."
hoptoad!farren       Garrison Keillor, "Lake Wobegon Days"

sl@van-bc.UUCP (Stuart Lynne) (07/11/87)

In article <7116@shemp.UCLA.EDU> srt@CS.UCLA.EDU (Matthew Merzbacher) writes:

>I've suggested before that the first type of moderation can be handled by
>forcing posting to source groups through a local (i.e., site) moderator,
>normally the new administrator.  This is an additional burden on the
>news admin, but since most sites rarely post more than one piece of source
>a month, it would hardly be overwhelming.

The only problem is the ever increasing number of one or two person sites. I
don't think I would have any problem getting my news adminstrator to let me
send out anything I wanted. Just a sec while I ask myself :-)

Seriously, with the cost of 286/386 machines, 3b1's on sale,
dcp/uuslave/uulink for pc's, mac's, amiga's and atari's; virtually anyone
who really wants to can get on the net looking like a normal uucp host.
There is currently no way to differentiate these small machines from larger
ones. 

If experienced users on larger machines cannot be trusted to post non
sources, I don't see how these new users can be.

I do think that multiple moderators is an interesting idea though and have
suggested that in the past. 

But with respect to the current moderator of comp.sources.misc, he DOES seem
to be getting things out with a minimum of delay. And is very responsive to
problem postings. He just recently was able to cancel the newguys posting's
before they had been on my machine for a day because of copyright
infringement problems. I would expect that while AT&T (for example) might
get very upset at people sending out their source code by mistake, prompt
actions like this will go a long way to keeping them from taking any
draconian measures against Usenet.

It's been said before, but I'll say it again. Lets give comp.sources.misc a
chance. If it isn't working well in a few months then bitch and complain.



-- 
{ihnp4!alberta!ubc-vision,uunet}!van-bc!Stuart.Lynne Vancouver,BC,604-937-7532

sl@van-bc.UUCP (Stuart Lynne) (07/13/87)

In article <6657@steinmetz.steinmetz.UUCP> jesup@mizar.UUCP (Randell Jesup) writes:
>		Mail to which Moderator?
>			(1) page@ulowell.cs.ulowell.edu (shortest path)
>			(2) jesup@ge-crd.arpa
>			(3) anyman@anyunix.uucp

Good idea!

>	It could have different moderators for different parts of the net
>(hard), or could try to estimate which is the easiest to reach (not easy),
>or not bother, in deciding the ordering.  That way, when moderators go on
>vacation, the users could route to the others, etc.  The moderators could

Alternatively for groups where a second moderator is not acceptable (ie,
mag.otherrealms, risks, arms-d etc) the moderator could set up a forwarding
address to some live person to either act as an interim moderator or at
least send a reply that the message was received and will be acted on when
the moderator gets back on such and such a date.

I believe that comp.sources.misc has already setup and alternate person at
his site to handle things when he is unavailable.


-- 
{ihnp4!alberta!ubc-vision,uunet}!van-bc!Stuart.Lynne Vancouver,BC,604-937-753 

page@ulowell.cs.ulowell.edu (Bob Page) (07/13/87)

jesup@mizar.UUCP (Randell Jesup) wrote:
>		Mail to which Moderator?
>			(1) page@ulowell.cs.ulowell.edu (shortest path)
>			(2) jesup@ge-crd.arpa
>			(3) anyman@anyunix.uucp

This may cause more trouble than it solves ... it means users get to
pick.  As much as possible, the software should do the work.  Most
sites don't even have a moderator's list; they just forward it to
some backbone site that sends it to the moderator.  If we go back to
the old way of "everyone maintains their own moderators file" we'll
probably (as a net) get worse performance, since many, many admins
don't update their moderators files.

A modification of the 'checkgroups' message could be cooked up to
sync moderators, but many UUCP sites don't run auto-routing mailers;
they'd require full bang path routing from here to there.

However, your proposal is worth considering from the point of multiple
moderators.  For example, news.lists is sent to two moderators,
usenet@gatech.edu and rick@seismo.css.gov ... it would mean some
coordinating work had to be done by the moderators, but would reduce
delays in reposting the message, and would provide multiple mail paths
to the moderators, increasing reliability of delivery.

The best part about this: it can be done NOW, with no changes to the
software.  The user doesn't have to learn anything new, either.  It's
an interim solution, but it would help.

Now it just means a group moderator has to agree to 'share' the duties.
I don't know if you'd get many, but that's a different problem.

..Bob
-- 
Bob Page, U of Lowell CS Dept.   page@ulowell.{uucp,edu,csnet} 

wayne@fmsrl7.UUCP (07/15/87)

	I have given the moderation problem a bit of thought and think I
may have a solution.  Since I am not a moderator, nor am I a backbone site,
I expect there to be flaws in this.  It is an initial proposal and needs
work to complete.

	When a user posts an article to a "moderated" group, the article 
is actually posted to an unmoderated group that a limited number of sites
carry.  This would flood a small number of sites with the article, giving
it a better chance to make it to the moderator.  The moderation approved
message could then cause the article to be transferred from the one group
to the other, maintaining the article ID.  At this point it would go out
as a "normal" moderated article.

	Benefits:  
	1) Any site that is willing to carry the traffic can have
	access to the unmoderated information.
	2) The moderator will have a better chance of getting the article.
	3) Anyone will be able to post to the unmoderated group.
	4) The posting interface remains constant.

	Problems:
	1) Software will have to be rewritten.
	2) Noise level may go up in some groups.
	3) People will have to understand that there is an unmoderated
	newsgroup and a seperate, collected, subset group.
	4) Every moderated group would require an associated unmoderated
	group.

Think it over.  Comment.  Critique.

	Looks like it solves most of the problems I've seen people complain
about.  The noise will be in the unmoderated group, not in the moderated one.
Readers can get ALL important info by reading the moderated group (not
worrying about missing stuff) but anyone may look at the unmoderated if
s/he so chooses.

/\/\ \/\/
-- 
Michael R. Wayne   ***   TMC & Associates   ***  Arpa: wayne@ford-vax.arpa
uucp: {philabs | pyramid} !fmsrl7!wayne   OR   wayne@fmsrl7.UUCP

jesup@steinmetz.steinmetz.UUCP (Randell Jesup) (07/16/87)

In article <991@van-bc.UUCP> sl@van-bc.UUCP (Stuart Lynne) writes:
...
>I do think that multiple moderators is an interesting idea though and have
>suggested that in the past. 
>
>But with respect to the current moderator of comp.sources.misc, he DOES seem
>to be getting things out with a minimum of delay.
...
>It's been said before, but I'll say it again. Lets give comp.sources.misc a
>chance. If it isn't working well in a few months then bitch and complain.
>
>{ihnp4!alberta!ubc-vision,uunet}!van-bc!Stuart.Lynne Vancouver,BC,604-937-7532

I have no qualms with comp.sources.misc, most of my problems are with other
sources/binaries and a few talk groups.  Not to denigrate him in any way,
or to imply it was his fault (I don't know), but for some time 
comp.sources.amiga was a black hole, eventually people got fed up and 
started posting sources to comp.sys.amiga again.  Any moderator is bound to
be unable to fufill his duties at least occasionally, lose his net 
connection, have hardware trouble, have a deadline, etc.  Also having
multiple moderators helps split the burden (unless the moderator is
a stickler for having COMPLETE control over everything in the group - which
they could have, no one is suggesting that every group needs multiple
moderators.)

	Randell Jesup
	jesup@ge-crd.arpa
	jesup@steinmetz.uucp

allbery@ncoast.UUCP (Brandon Allbery) (07/17/87)

As quoted from <1003@van-bc.UUCP> by sl@van-bc.UUCP (Stuart Lynne):
+---------------
| Alternatively for groups where a second moderator is not acceptable (ie,
| mag.otherrealms, risks, arms-d etc) the moderator could set up a forwarding
| address to some live person to either act as an interim moderator or at
| least send a reply that the message was received and will be acted on when
| the moderator gets back on such and such a date.
| 
| I believe that comp.sources.misc has already setup and alternate person at
| his site to handle things when he is unavailable.
+---------------

True, but comp.sources.misc is hand;ed differently from the majority of
moderated groups.  It's easy to have seconds when your job is to trash
non-sources and get the stuff out _pronto_, but a second moderator for
rec.arts.movies.reviews or rec.mag.otherrealms would (I should hope) take
some thought.
-- 
[Copyright 1987 Brandon S. Allbery, all rights reserved] \ ncoast 216 781 6201
[Redistributable only if redistribution is subsequently permitted.] \ 2400 bd.
Brandon S. Allbery, moderator of comp.sources.misc and comp.binaries.ibm.pc
{{ames,harvard,mit-eddie}!necntc,{well,ihnp4}!hoptoad,cbosgd}!ncoast!allbery
<<The opinions herein are those of my cat, therefore they must be correct!>>