andrew@calvin.doc.ca (Andrew Patrick) (12/12/90)
With all the recent discussions about the slow rate of postings to comp.sources.unix and the unresponsiveness of Rich Salz, I think it is time to think seriously about an alternative sources group. Thus, I would like to conduct a survey to determine if a PEER REVIEWED sources group is of interest to people, and feasible to operate. First, let me outline how I think a peer reviewed sources group might work. Then, I will ask people who are interested in participating in such a group to drop a note to a special address, and I'll compile the results and summarize to the net. We will then have to decide if we want to go through the procedure of creating the group. What I want to propose is a sources newsgroup that is based on a peer review system similar to that used for academic journals. For the sake of labelling, I will call this new newsgroup COMP.SOURCES.REVIEWED (perhaps comp.sources.unix.reviewed would be more appropriate -- it depends on what people are interested in). This newsgroup would be moderated by a committee that has 3 levels: Moderator, Associate Moderators, and Peer Reviewers. Here is a quick outline of what the roles of these players would be (again, based on the model provided by academic journals). Moderator The Moderator would be the adminstrator/delegator position in the organization. He (or she) would receive submission and assign them to the Associate Moderators. He would work with the Associate Editors in making decisions about what sources should be posted and would perform the actual postings once the software was reviewed and accepted. In general, this person would act as the "front-end" to the system -- handling questions, complaints, requests to repost, etc. In addition, the Moderator would have to maintain a database of Peer Reviewers who could be called upon by the Associate Moderators to evaluate software. (An "Yes" I would be willing to take this position if that is our best option. Gulp!) Associate Moderators The Associate Moderators would receive submission from the Moderator and assign them to Peer Reviewers. They would be responsible for ensuring that the reviews are returned in a timely fashion, and for making recommendations to the Moderator about the suitability/quality of the software. The Associate Moderators would also be responsible for preparing replies to the submitters that summarize the reviews, whether the sources were accepted or not. Peer Reviewers The Reviewers would receive sources from the Associates and evaluate them for their suitability, quality, etc. They would be responsible for preparing timely evaluations of the software, and perhaps providing expert advice to assist in the decisions made by the Moderators. So, the question I have for you is: Would you be interested in any one of these positions? I would like to collect a head count of the number of people who are interested in this kind of a news group, and at what level. If the numbers are large enough (say 5 Associates, 50 Reviewers), then I think we might want to talk about creating the group. If you are interested in participating, please drop a note to the following address explaining what positions are of interest (Moderator, Associate Moderator, or Peer Reviewer). You can also include a brief description of the skills you might bring to the job (e.g., what kinds of sources you could evaluate). Please send you mail to: reviewed@calvin.doc.ca (Internet, Bitnet, UUCP) If you have problems, you can try: reviewed%calvin@dgbt.doc.ca or reviewed%calvin@dgbt.crc.dnd.ca I will summarize the results to the net in about 1 week. -- Andrew Patrick, Ph.D. Department of Communications, Ottawa, CANADA andrew@calvin.doc.CA andrew@doccrc.BITNET HDTV: higher resolution, improved colour, wider screen, "sit-com" reruns. What's wrong with this picture?
tneff@bfmny0.BFM.COM (Tom Neff) (12/12/90)
In article <44190@cci632.UUCP> tvf@cci632.UUCP (Tom Frauenhofer) writes: >On the other hand, I believe that Mr. Salz is being unfairly toasted >by this flame-bait. Mr. Salz has been doing a fine job. Sure, I wish >he could get stuff out faster, but it does get out eventually. And >there are alternate sources groups for those who can't wait. Just out of curiosity, what would it look like if he were doing a NON-FINE job? Does such a concept exist, or is any level of performance automatically 'fine'? Just wondering.
tvf@cci632.UUCP (Tom Frauenhofer) (12/12/90)
This is being done to a small extent on comp.os.minix, where they have a referee's group that is supposed to take sources and evaluate them before you post them to the net. My only experience with this (I submitted some patches to the Minix UUCP program) has been that they are quite fast at reviewing the stuff and getting feedback back to you. I think peer review/referee-ing is superior to single moderator (single moderator = potential single point of failure). On the other hand, I believe that Mr. Salz is being unfairly toasted by this flame-bait. Mr. Salz has been doing a fine job. Sure, I wish he could get stuff out faster, but it does get out eventually. And there are alternate sources groups for those who can't wait. Disclaimer: I do not know Mr. Salz personally, I am just a happy "consumer" of the stuff he publishes in his moderated newsgroup. -- Thomas V. Frauenhofer, WA2YYW, tvf@cci.com | "Those who can, do; {uupsi,ccicpg}!cci632!tvf@uunet.uu.net | Those who can't, teach; tvf@frau.UUCP | Those who can't teach, tvf1477@ma.cs.rit.edu | flame." - anonymous
andrew@calvin.doc.ca (Andrew Patrick) (12/13/90)
In article <44190@cci632.UUCP> tvf@cci632.UUCP (Tom Frauenhofer) writes: ... >On the other hand, I believe that Mr. Salz is being unfairly toasted >by this flame-bait. Mr. Salz has been doing a fine job. Sure, I wish >he could get stuff out faster, but it does get out eventually. And >there are alternate sources groups for those who can't wait. Please note that I am not suggesting that we do away with comp.sources.unix, or Rich. What I am suggesting is that we consider exploring other methods of running sources groups, and one method to consider is peer review. I simply want to find out if enough people are interested in a peer reviewed newsgroup to make it feasible. Remember, people who are interested in participating in a peer reviewed sources group (comp.sources.reviewed?) should mail a note to reviewed@calvin.doc.ca stating their interest. -- Andrew Patrick, Ph.D. Department of Communications, Ottawa, CANADA andrew@calvin.doc.CA andrew@doccrc.BITNET HDTV: higher resolution, improved colour, wider screen, "sit-com" reruns. What's wrong with this picture?
andrew@calvin.doc.ca (Andrew Patrick) (12/20/90)
In article <1990Dec12.051059.3313@rick.doc.ca> I wrote: >With all the recent discussions about the slow rate of postings to >comp.sources.unix and the unresponsiveness of Rich Salz, I think it is >time to think seriously about an alternative sources group. Thus, I >would like to conduct a survey to determine if a PEER REVIEWED sources >group is of interest to people, and feasible to operate. ... >I would like to collect a head count of the number >of people who are interested in this kind of a news group, and at what >level. If the numbers are large enough (say 5 Associates, 50 >Reviewers), then I think we might want to talk about creating the group. ... After 1 week, I have received 24 replies by e-mail. Here is the count of people interested in the positions I described (note that the total count is > 24 since some people expressed interest in more than one position): Moderator: 2 Associate Moderator: 6 Peer Reviewer: 18 I also received some interesting comments on the proposal. Only one person thought the whole thing was impractical, but a number suggested that I may be proposing too much bureaucracy. In particular, they questioned whether the level of Associate Moderators was really needed. Many people suggested that it was important that reviews involve testing on a number of systems, and people often listed the systems that they had available. One person suggested that two special Associate Moderators be created to handle the archiving and patches/fixes. This sounds like a great suggestion. In general, I did not receive the number of replies that I had hoped for. They are still trickling in, however, so I will leave the mailbox open over the holidays and post a final summary in January. So as a reminder, if you are interested in the idea of a PEER REVIEWED sources group (comp.sources.reviewed?) based on an academic journal model, send a note to reviewed@calvin.doc.ca stating your interest. I am looking for people who would be interested in being Peer Reviewers, Associate Moderators, or Moderator. -- Andrew Patrick, Ph.D. Department of Communications, Ottawa, CANADA andrew@calvin.doc.CA andrew@doccrc.BITNET HDTV: higher resolution, improved colour, wider screen, "sit-com" reruns. What's wrong with this picture?
andrew@calvin.doc.ca (Andrew Patrick) (01/24/91)
A while ago I asked people who were interested in a PEER REVIEWED sources group to send me a note. This news group would distribute sources that had been evaluated by a group of Peer Reviewers drawn from a list of volunteers. The organization of the group would be modelled after scientific journals where submissions are sent to a Moderator, he/she assigns them to an Associate Moderator, who asks volunteers to evaluate the submission and report back. If the submission was found to be acceptable, it would be published along with the reviews provided by the volunteers. If it was not found to be acceptable, the author would be provided with feedback from the reviewers. I asked if people would be interested in acting as Moderator, Associate Moderator, or Peer Reviewer. I received replies from 39 people, and here is the final tally: Moderator 2 Associate Moderator 8 Peer Reviewer 31 (Some people expressed interest in more than one position.) Most people really liked the idea and had a number of suggestions. I feel sufficiently encouraged to pursue it further. So, in an upcoming posting you will find a formal CALL FOR DISCUSSION prepared according to the published guidelines for creating a new group. -- Andrew Patrick, Ph.D. Department of Communications, Ottawa, CANADA andrew@calvin.doc.CA andrew@doccrc.BITNET Bill Watterson for President!