brnstnd@kramden.acf.nyu.edu (Dan Bernstein) (04/12/91)
In article <1991Apr11.023044.2643@rick.doc.ca> csr@calvin.doc.ca (Andrew Patrick) writes: > We will attempt to handle sources that run on all the major computers > and operating systems. What about software that only runs on certain major computers and operating systems? What's a ``major computer'' anyway? If this group is only going to accept software that works under BSD, System V, MS-DOS, the Mac OS, VMS, and VM/CMS (just to name a few), it won't get many submissions. > Please use an informative "Subject" line when mailing postings. > Something like "nlist - New file list utility, Part01/04" Uh-oh. I can see the first few packages coming through now... uname - User-controlled name service sh - Clone of ``mesg n'' time - cute X clock with Japanese numbers. Plays Tetris, too! sh - System handler ksh - bouncy ball made out of lots of strands of rubber pwd - Protocol for Walking Dogs > For very large submissions, authors may wish to contact the moderator > (at "csr@calvin.doc.ca") to arrange an FTP file transfer. That should be happening anyway. > Each submission should include the following types of files. ``Types of files''? You mean not necessarily under those names? [ README, Makefile, INSTALL, man page ] > For large submissions, it is often useful to make subdirectories like > "man", "doc", "source", etc. What a leap in technology. One would think that such a progressive group would at least try to advance the minimum level of software it accepts. > but many users prefer that they be able to make > and test software in the current directory before they install it. Is there anyone who doesn't prefer this? I really like packages that have a separate installation script; make does not work well for files in different directories, and I hate having privileged operations hidden inside a Makefile. > The preferred form for patches is "diff" > format, using the "-c" option to produce context diff files. The preferred form should be unidiff. > - if submission is accepted: > - moderator discusses evaluations with author > - moderator posts sources and evaluations This is supposed to be modelled on traditional journal reviews, but reviewers don't get anonymity, and reviews are published. What a joke. > We are discouraging making repairs to submissions during the review > process. Ahahahahaha. That's funny. > Once the reviews are completed, you will receive a summary from the > moderator, and, if necessary, will have a chance to make repairs to > your package. So will the package be re-reviewed after that? > If you submit a package to CSR, you will be invited to become a > reviewer. Ooh, that tempts me soooo much. ---Dan
djm@eng.umd.edu (David J. MacKenzie) (04/12/91)
In article <9979:Apr1122:30:4291@kramden.acf.nyu.edu> brnstnd@kramden.acf.nyu.edu (Dan Bernstein) writes: If this group is only going to accept software that works under BSD, System V, MS-DOS, the Mac OS, VMS, and VM/CMS (just to name a few), it won't get many submissions. I don't follow that. Don't most Internet users use those systems most of the time, and most submissions to existing groups run on those systems? > Please use an informative "Subject" line when mailing postings. > Something like "nlist - New file list utility, Part01/04" Uh-oh. I can see the first few packages coming through now... Yes, please rename submissions that have names that conflict with existing programs that do something different! > Each submission should include the following types of files. ``Types of files''? You mean not necessarily under those names? Some authors prefer ReadMe, Install, or readme.doc, install.doc . . . doesn't make much difference, does it? (Well, uniformity is nice if you want to find all the documentation at a glance.) This point should be clarified, though, whichever way. > but many users prefer that they be able to make > and test software in the current directory before they install it. Is there anyone who doesn't prefer this? I really like packages that have a separate installation script; make does not work well for files in different directories, and I hate having privileged operations hidden inside a Makefile. make -n doesn't work well enough? It seems to be sufficient for me. I agree that it's nice to be able to test something without having to install it. Complex packages with lots of support files should probably support an environment variable or similar mechanism specifying the root(s) of the directories they use. Otherwise you have to build it twice -- one time configured for a test location, then again configured for the final installation location. > The preferred form for patches is "diff" > format, using the "-c" option to produce context diff files. The preferred form should be unidiff. Good idea. It would save a lot of bandwidth over time. The unidiff-aware patch program is easy to get from lots of ftp and uucp archive sites. Or it could even be posted to the newsgroup. Maybe GNU diff too. > - if submission is accepted: > - moderator discusses evaluations with author > - moderator posts sources and evaluations This is supposed to be modelled on traditional journal reviews, but reviewers don't get anonymity, and reviews are published. What a joke. I don't recall whether the group's charter mentions this issue. I don't know how closely the group ought to follow the traditional journal review model; what is most appropriate for this medium? > We are discouraging making repairs to submissions during the review > process. Ahahahahaha. That's funny. I agree; I think having the reviewers give the authors feedback, and getting fixes from the authors for problems they find, would probably speed up and improve the reviewing process significantly. > Once the reviews are completed, you will receive a summary from the > moderator, and, if necessary, will have a chance to make repairs to > your package. So will the package be re-reviewed after that? That could entail a long cycle of submit-review-fix-submit-review-fix etc. if authors aren't allowed to interact with reviewers. Consider the reviewers "post-beta" testers, and the submissions will probably make it through the pipeline more quickly, without loss of quality. Maybe with a gain in quality. Of course, that would eliminate the reviewers' anonymity, unless you interpose the group's moderators as middlemen in all communication between authors and reviewers, stripping "From: " lines before passing on the reviewers' comments. -- David J. MacKenzie <djm@eng.umd.edu> <djm@ai.mit.edu>
scs@iti.org (Steve Simmons) (04/12/91)
brnstnd@kramden.acf.nyu.edu (Dan Bernstein) writes: >[[ a bunch of stuff not worth repeating ]] Criticism will be much more effective if based on how we actually *do* things rather than worst possible interpretations of guidelines. Give us time to actually review things -- you'll probably like what you see. -- "Our informal mission is to improve the love life of operators worldwide." Peter Behrendt, president of Exabyte. Quoted in Digital Review, Feb 4, 1991.
brnstnd@kramden.acf.nyu.edu (Dan Bernstein) (04/13/91)
In article <scs.671464196@hela.iti.org> scs@iti.org (Steve Simmons) writes: > Criticism will be much more effective if based on how we actually *do* > things rather than worst possible interpretations of guidelines. Give > us time to actually review things -- you'll probably like what you > see. Oh? I still don't even know what you mean by ``review.'' Will the reviews be published? Journals don't publish referee reports. You claim to have modelled the process on what journals do. Your original rules say you will publish reviews. Apparently you're waffling internally. Yes or no? Was there any hint of this in the CFV? Will the reviews include lots of opinions on usability, features, etc.? I simply assumed not, and I sure don't remember anything like that from the original proposals. Journal reports, of course, are generally short, and always focus on whether the article is publishable. But now I'm told in e-mail that the c.s.r reviews may end up like magazine reviews. Yes or no? Was there any hint of this in the CFV? I could continue, but I hope you get the point. comp.sources.reviewed was not modelled upon journal publication except in the most superficial sense, and after you've finished making up rules for it, it may be just a shadow of what people expected when they voted for a group by that name. If you dropped the pretenses of imitating journals or of obeying the original charter, I wouldn't be so annoyed---but you also wouldn't have a newsgroup. ---Dan
csr@calvin.doc.ca (Andrew Patrick as CSR Moderator) (04/14/91)
In article <16705:Apr1306:15:1091@kramden.acf.nyu.edu> brnstnd@kramden.acf.nyu.edu (Dan Bernstein) writes: >Oh? I still don't even know what you mean by ``review.'' > >Will the reviews be published? Journals don't publish referee reports. >You claim to have modelled the process on what journals do. Your >original rules say you will publish reviews. Apparently you're waffling >internally. Yes or no? Was there any hint of this in the CFV? Nothing has changed. CSR will publish the reviews along with the sources. The only internal discussion in this area has been on whether reviewers will be allowed not to remain anonymous, and contact the authors of a submission for information or clarification. The result of that discussion is that reviewers will remain anonymous, unless they choose to reveal themselves directly to the authors (which is what occurs with journal reviews). The reviews that are posted with the software will not be attributed to an individual, which is different than the policies of some journals. On the other hand, journals I have published in often take a summary of the reviews and publish that in an introductory editorial, without attributing the opinions to any one individual. Futher, the authors of a submission will have a chance to see and address the reviews before they are posted with their submission, which is not the case for these introductory editorials. >Will the reviews include lots of opinions on usability, features, etc.? >I simply assumed not, and I sure don't remember anything like that from >the original proposals. Journal reports, of course, are generally short, >and always focus on whether the article is publishable. But now I'm told >in e-mail that the c.s.r reviews may end up like magazine reviews. Yes >or no? Was there any hint of this in the CFV? The reviews should contain information on how the package was tested, why it was found to be useful, what limitations were found, etc. >I could continue, but I hope you get the point. comp.sources.reviewed >was not modelled upon journal publication except in the most superficial >sense, and after you've finished making up rules for it, it may be just >a shadow of what people expected when they voted for a group by that >name. If you dropped the pretenses of imitating journals or of obeying >the original charter, I wouldn't be so annoyed---but you also wouldn't >have a newsgroup. Nothing has changed. Using a process similar to journal articles, CSR will operate by have submissions reviewed by a group of peers. There are some journals that do publish reviews along with the submissions, and that is what will happen in CSR. -- Andrew Patrick acting as Comp.Sources.Reviewed Moderator Department of Communications, Ottawa, CANADA csr@calvin.doc.CA
brnstnd@kramden.acf.nyu.edu (Dan Bernstein) (04/16/91)
In article <1991Apr14.025940.1747@rick.doc.ca> csr@calvin.doc.ca (Andrew Patrick as CSR Moderator) writes: > Nothing has changed. CSR will publish the reviews along with the > sources. There was no hint of this in the CFV. Out of 100 random journals, I'd be surprised if you found one that published articles. I consider it quite likely that a large number of voters simply *assumed* the reviews would not be published. I may be wrong, so I suggest that you poll the ``yes'' voters and find out what they really were expecting. That will settle the issue for good. ---Dan
csr@calvin.doc.ca (Andrew Patrick as CSR Moderator) (04/16/91)
In article <12060:Apr1602:04:5091@kramden.acf.nyu.edu> brnstnd@kramden.acf.nyu.edu (Dan Bernstein) writes: >In article <1991Apr14.025940.1747@rick.doc.ca> csr@calvin.doc.ca (Andrew Patrick as CSR Moderator) writes: >> Nothing has changed. CSR will publish the reviews along with the >> sources. > >There was no hint of this in the CFV. Out of 100 random journals, I'd be >surprised if you found one that published articles. I consider it quite >likely that a large number of voters simply *assumed* the reviews would >not be published. I may be wrong, so I suggest that you poll the ``yes'' >voters and find out what they really were expecting. That will settle >the issue for good. I don't want to prolong this discussion, but I cannot let errors of fact go by. Here is the section from the original Call For Votes: ... > If the Moderator and Peer Reviewers judge a submission to be acceptable, > the sources will be posted along with the written comments provided by > the Reviewers. ... This is a bit more than a hint! I imagine that the voters read the CFV before they voted, and expected exactly what was written. This is what they are going to get. As for examples of "real" journals that publish reviews, go to your university library and look up "Behavioural and Brain Sciences", published by Cambridge University Press. -- Andrew Patrick acting as Comp.Sources.Reviewed Moderator Department of Communications, Ottawa, CANADA csr@calvin.doc.CA
bart@dogmatix.cs.uoregon.edu (Barton Christopher Massey) (04/18/91)
In article <1991Apr16.034810.8005@rick.doc.ca> csr@calvin.doc.ca (Andrew Patrick as CSR Moderator) writes: ... > As for examples of "real" journals that publish reviews, go to your > university library and look up "Behavioural and Brain Sciences", > published by Cambridge University Press. Unfortunately, Andrew, this is non-responsive. Read your quoted material again: > In article <12060:Apr1602:04:5091@kramden.acf.nyu.edu> > brnstnd@kramden.acf.nyu.edu (Dan Bernstein) writes: ... > > Out of 100 random journals, I'd be > >surprised if you found one that published articles. A remarkable assertion indeed, and one I'd be willing to bet my life savings against!! :-) :-) Seriously, Dan, as a long-time net reader and poster (almost 10 years), allow me to describe to you the method I now almost invariably use when posting a rebuttal to someone else's Usenet article: 1) Before I post, I re-read my article for typos, bad grammar, and factual errors. I correct as many of them as I can find. 2) I then re-read the article again for tone. I attempt to make it conform to normal social standards of politeness. 3) I then leave the article unposted overnight. 4) In the morning, I re-read it a 3rd time, and decide if it's really what I meant to say. If not, I don't post it at all. This method, similar to the "Emily Postnews" recommendations, has helped me feel much more secure about the quality and tone of my postings in recent years. If you are not currently doing something like this, please give it a try yourself, both for your sake and for ours. Bart Massey bart@cs.uoregon.edu
Andrew Patrick <csr@calvin.doc.ca> (04/27/91)
Submitted-by: Andrew Patrick <csr@calvin.doc.ca>
Posting-number: Volume 1, Info 3
Archive-name: Submit
Comp.Sources.Reviewed
Guidelines for Submissions
Version 1.3 (4/10/91)
Comp.sources.reviewed (CSR) is a moderated newsgroup for the
distribution of program sources that have been evaluated by peer review
(new reviewers are always welcome). This document presents some
guidelines for submitting sources to CSR, and may change from time to
time as we get more experience with the review process. Comments about
these guidelines are also welcome.
What sources are acceptable?
----------------------------
We will attempt to handle sources that run on all the major computers
and operating systems. The limiting factor is whether we can find
people who are willing to review sources on particular machines. We
are equipped to review sources for the major flavors of Unix, DOS, and
perhaps VMS. If you are in doubt, send a description of the program
and the kinds of systems it is intended for, and we will see if we can
find reviewers for it.
Where should sources be sent?
-----------------------------
Submissions to CSR should be mailed to "csr@calvin.doc.ca". Most news
software will automatically mail postings to moderated groups to the
moderator, which in this case has been defined as "csr@calvin.doc.ca".
Please use an informative "Subject" line when mailing postings.
Something like "nlist - New file list utility, Part01/04" is preferred
over something like "Submission to comp.sources.reviewed". Your
submission will be assigned a short "reference" name (<12 characters)
for the review process and for archiving, so you may wish to supply a
possible name in your Subject line.
For very large submissions, authors may wish to contact the moderator
(at "csr@calvin.doc.ca") to arrange an FTP file transfer.
What form should submissions be in?
-----------------------------------
The form of the submission will depend somewhat on the type of system
it is intended for. The format for Unix submissions is detailed
below, and the format for other systems (e.g., DOS) should follow
these as much as possible.
Submissions should be in the form of one or more "shar" (shell
archive) files. Programs to build shar files have been posted to the
net, and are available from the usual archive sites.
A description of the package should precede the first shar file
(either in the first mailing, or as a separate letter). This
description should contain a description of the software package like
that contained in the README file (see below). In fact, in many cases
this description can simply be a copy of the README file.
If you are not familiar with this form of submissions, contact the
moderator of comp.sources.reviewed and we will send you some templates
to get you started.
What should be included in the submission?
------------------------------------------
Each submission should include the following types of files.
README: a description of the software package. This description
should include:
- the purpose and value of the software (give details here)
- the types of systems it is intended for (e.g., BSD only,
Unix and DOS, etc.)
- any dependencies in the system (e.g., must have perl to
run)
- known limitations of the software
- the authors of the software (with e-mail addresses)
- the "patchlevel" of the software (see below)
- any copying or distribution conditions
Makefile: a standard control file for the make utility. Having an
"install" target in the Makefile (so users can type "make install"),
is often convenient, but many users prefer that they be able to make
and test software in the current directory before they install it.
It also often a good idea to include a "clean" target to clean out
the "core", "*.o", etc. files.
Variables that users may wish to change should be clearly identified
at the beginning of the Makefile. Also, it is convenient to have
the parameters for known systems included in the Makefile, but
commented out.
e.g., CFLAGS=-DBSD -DUSE_TERMCAP
#CFLAGS=-DSYSV -DUSE_TERMINFO
Installation (INSTALL): a description of the steps necessary to
install the software. This should include a description of any
changes a user might wish to make to handle local conditions, and a
description of what happens when the user types "make install".
Man Page: for Unix sources, you should have one or more documentation
files prepared in man page format. If necessary, you may also wish
to include separate documentation files. The man page should
describe how the software is to be run, what parameters and options
are available, and the functions the software provides.
Patchlevel.h: you should have some method of determining what version
of the software this is. The preferred method is to use a
"patchlevel.h" file that contains information about the version level
(e.g., #define PATCHLEVEL 1.1), and this information is then used in
the source files (e.g., printed in the usage statement).
When patches are applied (see below), the "patchlevel.h" file should
be patched to reflect the new version number.
It is often useful to use some form of a source code control
system (e.g., SCCS or RCS) to keep track of the different versions
of software. Using such a system, each file in a package can
contain the version number information.
For large submissions, it is often useful to make subdirectories like
"man", "doc", "source", etc.
How are patches handled?
------------------------
Patches to published software should to be sent to the usual
submission address ("csr@calvin.doc.ca") with the word "patch" some
where in the "Subject" line. Patches will be reviewed and distributed
as quickly as possible. The preferred form for patches is "diff"
format, using the "-c" option to produce context diff files.
Patches should be accompanied by a complete description of the changes
made to the software, and the reasons behind those changes. The patch
must update the version number contained in the "patchlevel.h" file.
What are the steps of the peer review process?
----------------------------------------------
A typical sequence of events for the review of sources is:
- software is mailed to moderator
- moderator acknowledges receipt
- moderator briefly checks software against guidelines listed above
- moderator sends software to reviewers
- reviewers build, install, run, and test the software
- reviewers return evaluations to moderator (based on guidelines
described an another document)
- moderator compiles evaluations and makes final publication decision
- if submission is accepted:
- moderator discusses evaluations with author
- moderator posts sources and evaluations
- if submission is not accepted:
- moderator sends evaluations to author
- moderator may suggest revision and resubmission, or posting
in another newsgroup
What if the reviewers find problems?
------------------------------------
We are discouraging making repairs to submissions during the review
process. Reviewers may contact you for information and clarification,
but we do not envision fixes being applied during the course of a
review.
Once the reviews are completed, you will receive a summary from the
moderator, and, if necessary, will have a chance to make repairs to
your package.
Do authors automatically become reviewers?
--------------------------------------------
If you submit a package to CSR, you will be invited to become a
reviewer. This means that from time-to-time you may be sent other
people's software for evaluation. You may refuse the invitation, but
you should realize that this group must have a set of qualified
reviewers in order to function.
--
Andrew Patrick acting as Comp.Sources.Reviewed Moderator
Department of Communications, Ottawa, CANADA
csr@calvin.doc.CA