[comp.mail.uucp] "peer review" style of moderation

rcd@ico.isc.com (Dick Dunn) (11/30/90)

In the midst of the r$ fray, mrm@sceard.Sceard.COM (M.R.Murphy) suggests:

> Another way (not the only way :-) is to handle moderation of source and binary
> groups in the same way that refereed journals handle a similar problem. Have the
> moderator farm out the submissions to a group of interested folk...

This strikes me as one of the few possibly-useful suggestions I've seen.  I
review journal articles fairly often, and I've seen that the system can
support a heavy load.  (To be fair, I *haven't* been in the editor's
chair.)  If you've got a good group of people available to draw on, you can
get folks with special interests who can pay particular attention to the
details of what they're reviewing.  You can get more than one person to
look at stuff, as a sanity check.  And you can smooth over busy periods for
the reviewers.  (Editor asks reviewer "Can you handle this?  If so, have it
back to me in n weeks."  No response is the same as a negative response,
meaning "no, can't do it now."  Editor keeps trying 'til there are
reviewers set up, then waits for response.  Editor's job is somewhat more
boring, on average...but much less demanding.)

To answer an obvious question that some of you are itching to reply/follow
up with, "yes, I would be willing to be such a reviewer."  That's one
reason I think it would work and might be worth a try--the reviewer's task
is one that I can see myself taking on.  (Plus it would probably be a
damned sight more interesting than some of the journal submissions I've
seen!:-)

OK, so let's ask:
	- Is there any glaring flaw in Murphy's idea?
	- How many folks would be willing to contribute at the "reviewer"
	  level?
-- 
Dick Dunn     rcd@ico.isc.com -or- ico!rcd       Boulder, CO   (303)449-2870
   ...Mr. Natural says, "Use the right tool for the job."

abeals@autodesk.com (Plu festu, uloj) (12/01/90)

Dick Dunn talks about a "reviewer" strategy to help out moderators.

Good idea, but still has a single point of failure.

A better way would be to have multiple moderators - each moderator gets
a copy of each submission.  If the moderator thinks it's worth sending out,
s/he drops a copy onto the newsgroup ****using a modified copy of the
original message-id**** [<- yo, important point there]. 

You might ask "What does this buy us?  Multiple copies of messages in
the newsgroup?" By using a *modified copy* of the original message-id,
for example, if the original message-id of the submission mail message
was "<123456@mintaka.bedford.com>", the modified message-id of the
posted message would be something like
"<comp.unix.sources-123456@mintaka.bedford.com>".  Thus, if multiple
moderators posted the same message, the news software would reject the
posting as a duplicate, as it would already have a message by that
message-id in its history database.

What this means is that if two moderators send out a copy of the same
source posting, some folks will see the posting as coming from one
moderator [and set of paths] and others will see it as coming from a
different moderator, by a different route.

In this scheme, so long as all the moderators don't get sick or go on
vacation at once, someone will take up the slack and postings will
continue to appear.

Simple, neat, elegant.

--
Andrew Scott Beals
abeals@autodesk.com

peter@ficc.ferranti.com (Peter da Silva) (12/01/90)

How about splitting the moderation by using a variant of the Usenet Oracle
software? The software exists, works, and has (at least for me) been very
good about not dropping messages into mail black holes.
-- 
Peter da Silva.   `-_-'
+1 713 274 5180.   'U`
peter@ferranti.com 

gs26@prism.gatech.EDU (Glenn R. Stone) (12/02/90)

In <abeals.659990424@melange> abeals@autodesk.com (Plu festu, uloj) writes:
>A better way would be to have multiple moderators - each moderator gets
>a copy of each submission.  If the moderator thinks it's worth sending out,
>s/he drops a copy onto the newsgroup ****using a modified copy of the
>original message-id**** [<- yo, important point there]. 

Ummm, what if the code needs changes?  Then you have multiple versions
floating around.  Doubleplusungood.

I do think some sort of referee system is a good way to go, and I might
be willing to run a package or two past my really funky compiler (IBM AIX3
on the RS/6000.... part BSD, part POSIX, part USG, and occasionally a
royal pain :( ..... if there's anything weird about the code at all, xlc'll
catch it).  Maybe not too tightly organized..... one thing about USENET, 
start throwing rules at it, and watch the flame wars begin.  

-- Glenn R. Stone (gs26@prism.gatech.edu)
high tech anarchy at work.....

scs@lokkur.dexter.mi.us (Steve Simmons) (12/02/90)

In <abeals.659990424@melange> abeals@autodesk.com (Plu festu, uloj) writes:

>A better way would be to have multiple moderators - each moderator gets
>a copy of each submission.

gs26@prism.gatech.EDU (Glenn R. Stone) writes:

>Ummm, what if the code needs changes?  Then you have multiple versions
>floating around.  Doubleplusungood.

No, you do it just like refereeing a papers -- requests for changes (or
maybe patches, too) are collected by the moderator and returned to the
original author.  The author makes and co-ordinates the revisions, possibly
in direct contact with the referee(s).  If the author doesn't want to
do so, fine -- let him submit elsewhere.
-- 
 "I was talking about what it takes to be a real critic, not a critic
  wannabe."   -- Mike Godwin, rec.arts.comics critic

mathisen@dali.cs.montana.edu (Jaye Mathisen) (12/02/90)

In article <abeals.659990424@melange> abeals@autodesk.com (Plu festu, uloj) writes:
>for example, if the original message-id of the submission mail message
>was "<123456@mintaka.bedford.com>", the modified message-id of the
>posted message would be something like
>"<comp.unix.sources-123456@mintaka.bedford.com>".  Thus, if multiple
>moderators posted the same message, the news software would reject the
>posting as a duplicate, as it would already have a message by that
>message-id in its history database.


All kinds of problems here...  Take the case of moderator A and B both
submit package X, however A does it in 5 parts, and B does it in 6...  Oops,
there's an extra part here.  Of course, you could add additional headers
to assist in taking care of this, but all in all it sounds like far more
trouble than it's worth.

sfreed@gauss.unm.edu (Steve Freed) (12/03/90)

In article <abeals.659990424@melange>, abeals@autodesk.com (Plu festu, uloj) writes:
> A better way would be to have multiple moderators - each moderator gets
> a copy of each submission.  If the moderator thinks it's worth sending out,
> s/he drops a copy onto the newsgroup ****using a modified copy of the
> original message-id**** [<- yo, important point there]. 

The big problem with this is you get multiple people doing the same thing. If 
you and I were moderators, and you were checking out package "pdq" which
contains 2 megs of shar'd files, and were just about to post it in the next
day or so, I would really want to know this before investing 5 or 10 hours of
my time checking out the package, duplicating your efforts.

Multiple moderators would have to be in constant communication to avoid
duplicating efforts.  I think the best way to handle this is to have
one main or head moderator who farms out pending sources to the other 
modorators. The method of doing this would be worked out with the
different moderators.

At some point next spring, I will probably be in a position where I could
be one of the moderators or possibly the head moderator, but possibly
Rich $alz would be able to continue on in such a capacity. As we all 
cary on like this, I have yet to see his view on the matter. Has anyone
had the courtesy to send him email to ask him what he thinks?

-- 

Thanks,
Steve.                    sfreed@ariel.unm.edu

lws@comm.wang.com (Lyle Seaman) (12/12/90)

abeals@autodesk.com (Plu festu, uloj) writes:

>A better way would be to have multiple moderators - each moderator gets
>a copy of each submission.  If the moderator thinks it's worth sending out,
>s/he drops a copy onto the newsgroup ****using a modified copy of the
>original message-id**** [<- yo, important point there]. 

> [ ... ]

>Simple, neat, elegant.

And excessively redundant.
Yeah, it works, but people aren't microprocessors.  What you're asking
me (as hypothetical moderator) to do is to go through all the effort
of checking/cleaning/etc a posting, only to have it tossed as a duplicate.
And it really can be quite a bit of work.

No thanks.  The coordination between moderators to avoid duplication of
effort (hence, waste) is very valuable.
-- 
Lyle                      Wang             lws@comm.wang.com
508 967 2322         Lowell, MA, USA       uunet!comm.wang.com!lws
             The scum always rises to the top.