andrew@calvin.doc.ca (Andrew Patrick) (02/28/91)
This is an official Call for Votes for the proposed newsgroup
"comp.sources.reviewed". "Comp.sources.reviewed" will be a moderated
newsgroup (moderator: Andrew Patrick [andrew@calvin.doc.ca]) with the
following charter:
"Comp.sources.reviewed" is a moderated newsgroup for the
distribution of program sources that have been subjected to a Peer
Review process. Similar to the process used for academic
journals, submissions are sent to a moderator who then sends the
sources to Peer Review volunteers for evaluation. The Reviewers are
asked to provided a timely evaluation of the software by compiling
and running it on their machine. If time does not permit them to
complete a review, they are responsible for asking the moderator to
select another reviewer. If the Moderator and Peer Reviewers judge
a submission to be acceptable, the sources will be posted along with
the written comments provided by the Reviewers. If a submission is
not found to be acceptable, the submitter will be provided with the
Reviewers' comments, and they will have the option of addressing
those comments and submitting the sources again.
Volunteers interested in reviewing sources for this newsgroup can
send a note to "reviewed@calvin.doc.ca" to be placed on the list.
Votes for and against this proposed group should be sent to
votes@calvin.doc.ca
Please specify in your Subject: line if you vote yes or no by including
the words "yes" or "no" somewhere in the line, and then include in the
body a clear statement of your preference:
e.g., "I vote YES for comp.sources.reviewed."
or
e.g., "I vote NO for comp.sources.reviewed."
The voting period will last from Feb. 28 1991 to March 31 1991.
Technical Information on Reaching "calvin.doc.ca"
-------------------------------------------------
"Calvin.doc.ca" is an Internet site that should be easily reachable
from any network. For those who need more detailed information:
The host "calvin.doc.ca" is directly connected to the Internet and uses
an MX forwarder called "dgbt.doc.ca" (also known as "dgbt.crc.dnd.ca")
(192.12.98.7).
"calvin.doc.ca" is also reachable by UUCP using a path like:
...uunet!mnetor!lsuc!nrcaer!dgbt!calvin!votes
--
Andrew Patrick, Ph.D. Department of Communications, Ottawa, CANADA
andrew@calvin.doc.CA
"The interface IS the program."kent@sparky.IMD.Sterling.COM (Kent Landfield) (03/01/91)
In article <1991Feb28.053121.7739@rick.doc.ca> andrew@calvin.doc.ca (Andrew Patrick) writes: >This is an official Call for Votes for the proposed newsgroup >"comp.sources.reviewed". "Comp.sources.reviewed" will be a moderated >newsgroup (moderator: Andrew Patrick [andrew@calvin.doc.ca]) with the >following charter: > > "Comp.sources.reviewed" is a moderated newsgroup for the > distribution of program sources that have been subjected to a Peer > Review process. Similar to the process used for academic > journals, submissions are sent to a moderator who then sends the > sources to Peer Review volunteers for evaluation. Question: What types of packages are you envisioning on reviewing ? Packages like elm, mush, perl, ... have their own beta test groups who usually do a solid job of testing their software. For that matter, any software that has a beta test team would probably not need or have any reason to post through this group. Question: Are you planning on accepting any type of software running on unix, dos, coherent, minix, acorn, amiga ...? Or will this be focused for a specific operational environment? How will you locate enough reviewers for packages that run in other than the major environments. Question: What happens when you cannot find enough people to review the package ? Is the submission rejected ?? Are we going to be seeing a lot of "requests for reviewers" on the net ? Question: This whole thing seems to have started as an emotional reaction to delays in postings to c.s.unix. What are the expected time delays in submissions through this group ?? Just the setup to evaluate a package by your description is going to take a bit of time. The review by multiple people will take time. [ Even the reviewers have real jobs... :-) ] If there are any questions, problem etc., that will take more time. My guess is that you are looking at a 45 to 60 day delay on the optimistic side. Doesn't this procedure usually take 6 to 9 months for submissions to be accepted and published in academic journals ?? I can see that there could be a use for this group but I don't think that there has been enough discussion for a real call for votes. In the discussion period I have seen a grand total of 5 articles on the topic and three of them have come from Andrew Patrick [andrew@calvin.doc.ca]. If we are going to do this, lets do it right. There has not been enough discussion to warrant a vote. My intention here is to try to get some information and to stimulate a discussion so that any vote is based on merit and not emotion... -Kent+ -- Kent Landfield INTERNET: kent@sparky.IMD.Sterling.COM Sterling Software, IMD UUCP: uunet!sparky!kent Phone: (402) 291-8300 FAX: (402) 291-4362 Please send comp.sources.misc-related mail to kent@uunet.uu.net.
dwh@ataritx.uucp (Dave Hanna) (03/01/91)
In article <1991Feb28.160555.8446@sparky.IMD.Sterling.COM> kent@sparky.IMD.Sterling.COM (Kent Landfield) writes: >In article <1991Feb28.053121.7739@rick.doc.ca> andrew@calvin.doc.ca (Andrew Patrick) writes: >>This is an official Call for Votes for the proposed newsgroup >>"comp.sources.reviewed". >Question: This whole thing seems to have started as an emotional reaction >to delays in postings to c.s.unix. What are the expected time delays in >submissions through this group ?? Just the setup to evaluate a package >by your description is going to take a bit of time. The review by multiple >people will take time. [ Even the reviewers have real jobs... :-) ] If >there are any questions, problem etc., that will take more time. My >guess is that you are looking at a 45 to 60 day delay on the optimistic >side. 45 to 60 days sounds like a reasonable guess. What are delays running through comp.sources.unix now? I want to be careful how I come into this debate. I appreciate the job that Rich $alz is doing, and that it's a tough job. I also appreciate that the comp.sources groups are not an "inalienable right" to which I am entitled, and am thankful for what does come out of them. On the other hand, I can appreciate the frustration of authors [and potential users] of a piece of software that submit it to comp.sources.unix and then wait for seemingly interminable and unpredictable periods of time and see nothing come out. I don't know what the actual delays from submission to posting are, because Rich doesn't choose to tell us what has been submitted and is in the queue. I know the monthly posting for Elm says that several patches have been submitted to comp.sources.unix, and that none have yet appeared. I've been using Elm for at least six months, and it was at patch level 6 when I started, so I would guess that some of those patches have been in the queue for many, many months. (And they have been beta tested before they are submitted.) The point of this is, if we can get a mechanism that will produce archiveable sources with a fairly predictable 45-60 day delay, I'm all for it. > -Kent+ >-- >Kent Landfield INTERNET: kent@sparky.IMD.Sterling.COM >Sterling Software, IMD UUCP: uunet!sparky!kent >Phone: (402) 291-8300 FAX: (402) 291-4362 >Please send comp.sources.misc-related mail to kent@uunet.uu.net. -- Dave Hanna Atari Microsystems Corp UUCP ...!texsun!letni!ataritx!dwh ...!ames!atari!dhanna
emv@ox.com (Ed Vielmetti) (03/01/91)
In article <1991Feb28.221653.10062@ataritx.uucp> dwh@ataritx.uucp (Dave Hanna) writes: The point of this is, if we can get a mechanism that will produce archiveable sources with a fairly predictable 45-60 day delay, I'm all for it. I'm reasonably sure that comp.sources.misc fits the bill right now. >Please send comp.sources.misc-related mail to kent@uunet.uu.net. And followups to news.groups,comp.sources.d. -- Msen Edward Vielmetti /|--- moderator, comp.archives emv@msen.com
karsten@tfl.dk (03/01/91)
In article <1991Feb28.160555.8446@sparky.IMD.Sterling.COM>, kent@sparky.IMD.Sterling.COM (Kent Landfield) writes: > In article <1991Feb28.053121.7739@rick.doc.ca> andrew@calvin.doc.ca (Andrew Patrick) writes: >>This is an official Call for Votes for the proposed newsgroup >>"comp.sources.reviewed". "Comp.sources.reviewed" will be a moderated >>newsgroup (moderator: Andrew Patrick [andrew@calvin.doc.ca]) with the >>following charter: >> >> "Comp.sources.reviewed" is a moderated newsgroup for the >> distribution of program sources that have been subjected to a Peer >> Review process. Similar to the process used for academic >> journals, submissions are sent to a moderator who then sends the >> sources to Peer Review volunteers for evaluation. > > Question: What types of packages are you envisioning on reviewing ? > Packages like elm, mush, perl, ... have their own beta test groups > who usually do a solid job of testing their software. For that matter, > any software that has a beta test team would probably not need or have > any reason to post through this group. Those packages submitted to the moderator would be reviewed if possible. > Question: Are you planning on accepting any type of software running > on unix, dos, coherent, minix, acorn, amiga ...? Or will this be focused > for a specific operational environment? How will you locate enough > reviewers for packages that run in other than the major environments. I know many would like the group to be one operating system only, but USENET can be carried on other operating systems. This is written on VAX/VMS. Currently we got a group for reviewed UNIX software, but none for other operting systems. There should be a group for other operating systems. Many of the bested public domain packages have been ported from UNIX to other operating systems, and some the other way. In the future, we can expect people will comply with the posix standards, which will supported on many operating systems. So in the future packages will not be for one operating system. For these reasons it would be stupid to limit the group to one or a few operating systems. > Question: What happens when you cannot find enough people to review the > package ? Is the submission rejected ?? Are we going to be seeing a lot > of "requests for reviewers" on the net ? There would be no alternative to rejecting such packages, and ask the submitter of the code to post it to some other group. My guess is that "reguest for reviewers" will be rare because most public domain software is written for a few popular operating systems. Andrew has already 20-30 reviewers. > Question: This whole thing seems to have started as an emotional reaction > to delays in postings to c.s.unix. What are the expected time delays in > submissions through this group ?? Just the setup to evaluate a package > by your description is going to take a bit of time. The review by multiple > people will take time. [ Even the reviewers have real jobs... :-) ] If > there are any questions, problem etc., that will take more time. My > guess is that you are looking at a 45 to 60 day delay on the optimistic > side. Doesn't this procedure usually take 6 to 9 months for submissions > to be accepted and published in academic journals ?? Journals use paper mail for transporting papers. After the paper has been reviewed, it must be typeset, and proff read. These things takes time. Besides, papers for conferences are normally reviewed in a few months. The moderator will need ca 1 week for assigning moderators and submitting the code to the reviewers. The reviewers could use one month, and the moderator one week for collecting the results and taking action. That's 6 weeks, and You must admit that both the moderator and the reviewers have been given enough time. It would of course take more time in case the code has to be corrected, and so what? Then the whole USENET won't vaste time on installing a collecting of bugs. > I can see that there could be a use for this group but I don't think that > there has been enough discussion for a real call for votes. In the > discussion period I have seen a grand total of 5 articles on the topic and > three of them have come from Andrew Patrick [andrew@calvin.doc.ca]. If > we are going to do this, lets do it right. There has not been enough > discussion to warrant a vote. My intention here is to try to get some > information and to stimulate a discussion so that any vote is based on > merit and not emotion... Perhaps. I have only seen a posting raising the first point of your posting, i.e., which operating systems should be supported. Have I missed some postings, or why didn't You raise the other points in the period of discussion? > -Kent+ > -- > Kent Landfield INTERNET: kent@sparky.IMD.Sterling.COM > Sterling Software, IMD UUCP: uunet!sparky!kent > Phone: (402) 291-8300 FAX: (402) 291-4362 > Please send comp.sources.misc-related mail to kent@uunet.uu.net.
chip@tct.uucp (Chip Salzenberg) (03/02/91)
[ Followups to news.groups. And alt.* has nothing to do with comp.*; leave them out of any further discussions, please. ] According to kent@sparky.IMD.Sterling.COM (Kent Landfield): >I can see that there could be a use for this group but I don't think that >there has been enough discussion for a real call for votes. A discussion period is mandated. A volume of discussion is not. You had your chance to talk; it has passed. Now it's voting time. -- Chip Salzenberg at Teltronics/TCT <chip@tct.uucp>, <uunet!pdn!tct!chip>
brnstnd@kramden.acf.nyu.edu (Dan Bernstein) (03/02/91)
In article <1991Mar1.082855.149@tfl.dk> karsten@tfl.dk writes: > Journals use paper mail for transporting papers. After the paper has > been reviewed, it must be typeset, and proff read. These things takes > time. And you're implying that since comp.sources.reviewed will use e-mail instead, it will run smoothly? Be serious. Referees have always been and will always be the biggest bottleneck in journal publication. ---Dan
kent@sparky.IMD.Sterling.COM (Kent Landfield) (03/03/91)
In article <1991Mar1.150759.11832@rick.doc.ca> andrew@calvin.doc.ca (Andrew Patrick) writes: > >NOTE: I am restricting this discussion to news.groups: Many people feel that this should be discussed in comp.sources.d since that is the area which we are discussing. For my postings, I added it back... I wrote in my initial posting and Andrew quoted... # My intention here is to try to get some # information and to stimulate a discussion so that any vote is based on # merit and not emotion... Looks like I stimulated some discussion... :-) Andrew writes: >I was about to let the whole thing lapse due to lack of discussion, but >postings and e-mail convinced me to post a CFV and see what happens. > >At this point, 69 people disagree with you and feel it is time for >voting -- they have already cast theirs. > >If you don't like the proposal, vote NO. When I initially responded to the CFV, I was acting in a manner that was informational in scope. By Andrew's own admission there was a lack of any real discussion on the topic. The questions I posed were to gather the background I needed to make an intelligent decision. Nothing more, nothing less. I don't vote for local, state or national issues without knowing what they are. Why should the net issues be any different... :-) Andrew has answered most of my inital questions and with the help of Warren Tucker, a better picture of just what the group is going to supply has now been presented. To snapshot what Warren wrote: | c.s.r would | | o provide a more reliable and organized way to find motivated tester-critics | o increase the number of cases and environments for testing | o improve the end product and reduce patch frequency | o more than compensate for delays in net.delivery. | o should be a comp.sources."new-and-augmenting" not a comp.sources. | "all-is-made-new" or comp.sources."death-to-$alz". I can see a group like this would fill a currently existing void for authors. I have run into the problem myself with rkive. I "recruited" a beta test team so that I would have others beating on the code in a different manner than I normally did. It was not easy to get it set up. This type of group could have saved me that initial setup... :-) An additional area that need clarification is how patches going to be dealt with. Will they go back through c.s.r for re-evaluation prior to being posted? Or will patches be accumulated prior to resubmission through c.s.r ? I would like to suggest that a more indepth description of the group's general procedures and proposed policy be posted. Yes, I know that this is beyond what is needed to vote but some of us think the vote is just a formality... :-) You could consider it a headstart on the introduction and policy posting ... :-) From a submitters perspective, how are authors to submit and work with the group, how will the group deal with the inevitable patches, and what are areas that would cause rejections are a good starting place. These are real areas that will have to be addressed. Now is as good a time as any... :-) :-) -Kent+ -- Kent Landfield INTERNET: kent@sparky.IMD.Sterling.COM Sterling Software, IMD UUCP: uunet!sparky!kent Phone: (402) 291-8300 FAX: (402) 291-4362 Please send comp.sources.misc-related mail to kent@uunet.uu.net.
tneff@bfmny0.BFM.COM (Tom Neff) (03/03/91)
I think we should call it comp.sources.reinvent-wheel, and require that
each submission
(a) perform some function we can already do with standard software;
(b) add enough frivolous bells and whistles to ensure that it will fail
on lots of platforms;
(c) be twice the size of the package it "replaces," or a minimum of 25
parts, whichever is bigger;
(d) include a vigorous schedule of fifteen-part patches at frequent
intervals, so that the frivolous bells and whistles can be
endlessly tinkered with and the set of unsupported platforms
constantly reshuffled; at least one run of three releases within a
one-month period is mandatory.
I urge advocates of a new source group to consider this proposal, as it
would provide a well-tailored niche for so many of today's offerings.
Under my scheme, alt.sources and comp.sources.misc would have to get by
with shorter, widely portable programs that actually do something new
and useful. It would be tough at first, but we'll bear the sacrifice
bravely.karsten@tfl.dk (03/03/91)
In article <4989:Mar123:50:0391@kramden.acf.nyu.edu>, brnstnd@kramden.acf.nyu.edu (Dan Bernstein) writes: > In article <1991Mar1.082855.149@tfl.dk> karsten@tfl.dk writes: >> Journals use paper mail for transporting papers. After the paper has >> been reviewed, it must be typeset, and proff read. These things takes >> time. > > And you're implying that since comp.sources.reviewed will use e-mail > instead, it will run smoothly? Be serious. Referees have always been and > will always be the biggest bottleneck in journal publication. > > ---Dan No I am implying that when it runs smoothly, it can be faster than journal publication. Noboby can tell in advance wheither or not the referees of comp.sources.reviewed would be bottleneck, but I think that the moderator of c.s.r could easily find a substitute a slack referee. Andrew has already 20-30 applicant for the posts as referees. I addition, the referees will work faster because they know the system could be fast. So is the human mind. No single person will have as much work with running c.s.r as R$ has with comp.sources.unix, even if c.s.r get a larger throughput, and it will be much easier to replace those who are slack. Karsten Nyblad TFL, A Danish Telecommunication Research Laboratory E-mail: karsten@tfl.dk
andrew@calvin.doc.ca (Andrew Patrick) (03/05/91)
In article <1991Mar3.035928.28569@sparky.IMD.Sterling.COM> kent@sparky.IMD.Sterling.COM (Kent Landfield) writes: >In article <1991Mar1.150759.11832@rick.doc.ca> andrew@calvin.doc.ca (Andrew Patrick) writes: >> >>NOTE: I am restricting this discussion to news.groups: > >Many people feel that this should be discussed in comp.sources.d since that >is the area which we are discussing. For my postings, I added it back... OK. ... >An additional area that need clarification is how patches going to be >dealt with. Will they go back through c.s.r for re-evaluation prior >to being posted? Or will patches be accumulated prior to resubmission >through c.s.r ? Given that as of one week ago I thought this proposal was not going to fly, I haven't done a lot of thinking on this. Also, I do not want to be responsible for setting all the policy for this group. You will be the users of the group -- those of you who submit sources and those of you who use them. Therefore, it should be you who makes these kinds of decisions. I can give you my opinions on these issues, and I encourage others, especially those who have agreed to be reviewers, to discuss what you want to see. Concerning patches, I envision enlisting the help of one or more associates who will be responsible for maintaining an archive and quickly testing and distributing patches. I imagine that when patches come in, they will be quickly tested against the software in the archive and then distributed ASAP. I also foresee complete re-posts of packages once a fair number of patches have been applied, especially if the posting traffic is light (e.g., while waiting for reviews of new sources to come in). >I would like to suggest that a more indepth description of the group's >general procedures and proposed policy be posted. Yes, I know that this >is beyond what is needed to vote but some of us think the vote is just a >formality... :-) You could consider it a headstart on the introduction >and policy posting ... :-) From a submitters perspective, how are authors to >submit and work with the group, how will the group deal with the inevitable >patches, and what are areas that would cause rejections are a good starting >place. These are real areas that will have to be addressed. Now is as good >a time as any... :-) :-) Again, I can only give you my opinions on these issues: Concerning the relationship between the author and the group of reviewers, I imagine it will go something like this: - author submits source to moderator - moderator assigns source to reviewers - reviewers return comments (based one some guidelines) to moderator - moderator makes posting decision based on reviews - if accepted: - moderator prepares summary of reviews to be attached to posting - moderator discusses summary with author - moderator posts sources and summary of reviews - if rejected: - moderator prepares summary of reviews to be returned to author - moderator discusses reviews with author - moderator may suggest that sources be revised and re-submitted - moderator may suggest alternate place for sources I do not foresee any direct contact between the reviewers and the author, and I think it might be a good idea for the reviewers to remain anonymous, but that is something to be discussed by you. Concerning criteria for acceptance, this will have to be worked out. Also, I think the criteria will have to evolve somewhat as we see what kinds of sources are submitted. Obviously, one criteria will have to be that it is a well-formed package: that is has README, Makefile, and patchlevel files, man pages, installation instructions, etc. The installation procedure is also an obvious area of evaluation. The sources will also have to be evaluated for portability, as much as possible. Did it compile on a variety of machines (at least those it was intended for)? This test for portability I see as one of the hardest things to conduct, and I am not sure how to do it. Obviously, selecting reviewers that run on a variety of platforms will be a first step. Another area of evaluation will have to be operation -- did it run smoothly, or are their obvious problems and bugs. Also, are their obvious shortcomings that could be fixed to produce a much better package. In summary, I don't know what I have gotten myself into (some would say a pile of SH*T), but I think it is an idea who's time has come. I hope that WE can work out the policy and operation of the group, and make it a valuable resource for both the authors and the users. -- Andrew Patrick, Ph.D. Department of Communications, Ottawa, CANADA andrew@calvin.doc.CA "The interface IS the program."
src@scuzzy.in-berlin.de (Heiko Blume) (03/06/91)
andrew@calvin.doc.ca (Andrew Patrick) writes: >I do not foresee any direct contact between the reviewers and the >author, and I think it might be a good idea for the reviewers to remain >anonymous, but that is something to be discussed by you. i think as a reviewer i certainly would send mail to the author sooner or later like "what's that supposed to do / is this intended" etc. perhaps a mailing list would be a good idea! >Obviously, one criteria will have to be that it is a well-formed >package: that is has README, Makefile, and patchlevel files, man >pages, installation instructions, etc. The installation procedure is >also an obvious area of evaluation. and _*DON't*_ forget revision numbers! doesn't tell me much if something is patchlevel 1, if i don't know of what revision. that's a pain in the ass for me all the time... perhaps using rcs from the beginning might be a win. >The sources will also have to be evaluated for portability, as much as >possible. Did it compile on a variety of machines (at least those it >was intended for)? This test for portability I see as one of the >hardest things to conduct, and I am not sure how to do it. Obviously, >selecting reviewers that run on a variety of platforms will be a first >step. i thought that was a main point in reviewing in the first place?! -- Heiko Blume <-+-> src@scuzzy.in-berlin.de <-+-> (+49 30) 691 88 93 public UNIX source archive [HST V.42bis]: scuzzy Any ACU,f 38400 6919520 gin:--gin: nuucp sword: nuucp uucp scuzzy!/src/README /your/home
rsalz@bbn.com (Rich Salz) (03/07/91)
Why is there a problem with having more than one source of source code posted to Usenet? We already have comp.sources.unix,comp.sources.misc,alt.sources. /r$ -- Please send comp.sources.unix-related mail to rsalz@uunet.uu.net. Use a domain-based address or give alternate paths, or you may lose out.
davidsen@crdos1.crd.ge.COM (Wm E Davidsen Jr) (03/08/91)
In article <1991Mar06.151615.3212@scuzzy.in-berlin.de> src@scuzzy.in-berlin.de (Heiko Blume) writes: | and _*DON't*_ forget revision numbers! doesn't tell me much if | something is patchlevel 1, if i don't know of what revision. | that's a pain in the ass for me all the time... | perhaps using rcs from the beginning might be a win. Sounds as if this group is going to be so picky that alt.sources will continue to be the main source of software. I fail to see why use of RCS makes the software better or worse than software using SCCS or any of the other programs which handle versions. Or even just a file called version which holds the information entered by a human being. I don't see that making use of RCS or any other package has any bearing on the quality of the software. | >The sources will also have to be evaluated for portability, as much as | >possible. Did it compile on a variety of machines (at least those it | >was intended for)? This test for portability I see as one of the | >hardest things to conduct, and I am not sure how to do it. Obviously, | >selecting reviewers that run on a variety of platforms will be a first | >step. Let's not get carried away on portability. A clear statement of required environment is fine, it doesn't have to run everywhere to be useful. A submission should be tested for BSD, SysV, and in most cases Xenix/386 and V.3.2/386 since they represent the largest number of users of a single type of UNIX. Compilers needs should be tested on K&R, ANSI, and gcc flavors of C. I would also support a requirement that all compile options be documented by comments in the Makefile. -- bill davidsen (davidsen@crdos1.crd.GE.COM -or- uunet!crdgw1!crdos1!davidsen) "Most of the VAX instructions are in microcode, but halt and no-op are in hardware for efficiency"