[gnu.emacs] Emacs Lisp Archives -- organization

dsill@RELAY.NSWC.NAVY.MIL (06/23/89)

Okay, I've given some thought to the organization of the elisp
archives.  I'll describe what I have in mind, then you can
flame/praise/comment upon it.

Everything I dumped under elisp-archive initially will be put in a
subdirectory called "own-risk", indicating that there are no
guarantees that it:
    -will run
    -is documented
    -won't damage anything on your system
In other words, use this code at your own risk.

Other subdirectories will be set up such as those suggested recently
by Chris Siebenmann:
    modes	- various major modes
    interfaces	- interfaces to other systems (eg dbx.el, mh-e.el)
		  perhaps with subdirectories for X, news, etc.
    packages	- bundles of functions that don't make a mode
    functions	- individual useful functions
    misc	- things that don't fit elsewhere
Any others?

I would place code under these directories only after I'm satisfied
that it is sufficiently safe and that its purpose, installation, and
use is adequately documented.  I would also require the author to
include the GNU General Public License.

I'd also change the "contact" field of the Lisp Code Directory
datafile to indicate the location of the code if it's in the archive.
This would allow an elisp-archive-retrieve command to be hacked up, as
well as indicating if an item is in the archive.

I think this approach would maximize the availability of the code
at a minimum of effort to the contributors, maintainers, and users.

One unresolved issue is how to name the archive files.  In most cases,
I can simply use the author's name.  Sometimes, however, the author
hasn't provided a name, or the name isn't unique, or the code consists
of modifications to a distributed package such as rmail.

Comments, please?

-Dave Sill
 dsill@relay.nswc.navy.mil
 elisp archive coordinator

worley@EDDIE.MIT.EDU (Dale Worley) (06/23/89)

    I would also require the author to
    include the GNU General Public License.

Why?

Since most of this code has been publically posted, it's in the public
domain, and applying conditions on its distribution (which *you* can
do yourself -- it's public domain, after all) is sort of pointless.

Also, while I like the fact that RMS is writing neat stuff, I'm sort
of annoyed with his idea that everyone is *obligated* to use Gnu-style
licensing.  Let's give people the freedom to decide the conditions
*they* want on distribution of their code.

Dale
--
Dale Worley, Compass, Inc.                      worley@compass.com
The War on Drugs -- Prohibition for the '80s.

sdr@flash.alliant.com (Stephen D. Rogers) (06/23/89)

== Archive Directory Structure ==

I have a small set of personal lisp packages (mostly modifications to
posted ones) arranged in a tree that is very similar to the one you are
proposing.  A difference that I consider important is that subdirectories
in my tree use category names that are singular, rather than plural.  My
rationale is best explained with an example.  Consider:

	(autoload 'C++-mode "mode/C++")

versus

	(autoload 'C++-mode "modes/C++")

I think that the first alternative better expresses the intent: I want to
autoload a C++ `mode', not a C++ `modes'.

Another way to describe the difference is in terms of focus.  I am
focused on *using* the individual lisp packages.  Thus I want to use all
of a package's relative pathname to describe it, not just the filename.
If I were only focused on *maintaining an archive* of lisp packages, then
the pathnames to individual lisp packages may not be as important as
their categorization.

I think that the same directory structure should be used for *archiving*
and for *using* lisp packages.  However, I also think that the directory
structure should be oriented towards package use first, and package
archiving second.

== Archive File Names==

I think that file names in the archive should be based on package names,
prefixed by the login name of the author.  If login names are duplicated,
add a digit to end of the login name.  The same prefix should be used for
the same author throughout the archive.  For example,

	(autoload 'C++-mode "mode/sdr-C++")

Would be used to access a C++ mode provided by me (sdr).  If some
other sdr submitted a package anywhere else in the archive, he/she
would be `sdr1' and so on.  For example, let's say sdr1 submitted
a new shell mode:

	(autoload 'shell-mode "mode/sdr1-shell")

If a package has several files associated with it, it should be
given its own subdirectory, which contains the associated files.
For example, let's say sdr1 submitted a set of special shell
modes that are built on top of his base shell mode.  The base
shell mode would still be accessed as

	(autoload 'shell-mode "mode/sdr1-shell")

and the special shell modes would be accessed as

	(autoload 'C-shell-mode      "mode/sdr1-shell/csh")
	(autoload 'Korn-shell-mode   "mode/sdr1-shell/ksh")
	(autoload 'Bourne-shell-mode "mode/sdr1-shell/sh")
	...

It would also be worthwhile to provide an AUTHORS file for the archive,
which provided a cross reference of author id, name, and e-mail address.
For example,

	sdr	"Stephen D. Rogers"	srogers@graphics.alliant.com

One last thought: If file name length is an issue, perhaps the author
specification should be an additional directory level, rather than
a prefix.  For example,

	(autoload 'shell-mode "mode/sdr1/shell")

Maybe better: we should just add the digit to the package name as
an extension.  For example,

	(autoload 'shell-mode "mode/shell.1")

In this case, the first field in the AUTHORS file should be the
relative package name instead of the author id.

kjones@talos.UUCP (Kyle Jones) (06/23/89)

Dave Sill writes:
 > Everything I dumped under elisp-archive initially will be put in a
 > subdirectory called "own-risk", indicating that there are no
 > guarantees that it:
 >     -will run
 >     -is documented
 >     -won't damage anything on your system
 > In other words, use this code at your own risk.

All the code in the archive should be considered "use at your own risk".
I don't think you should even attempt to verify that a package won't damamge
anything on anyone's system.  Emacs itself comes with no warranty, so you
can't gaurantee programs running under it.

 > Other subdirectories will be set up such as those suggested recently
 > by Chris Siebenmann:
 >     modes	- various major modes
 >     interfaces	- interfaces to other systems (eg dbx.el, mh-e.el)
 > 		  perhaps with subdirectories for X, news, etc.
 >     packages	- bundles of functions that don't make a mode
 >     functions	- individual useful functions
 >     misc	- things that don't fit elsewhere
 > Any others?

terms - like the lisp/term directory in the Emacs distribution.  I'd hope
	new term files would be migrated rapidly into Emacs distribution
	but the	archive would be a good interim place for them.

dsill@RELAY.NSWC.NAVY.MIL (06/24/89)

>From: compass!worley@eddie.mit.edu (Dale Worley)
>Why [require inclusion of GNU license]?
>
>Since most of this code has been publically posted, it's in the public
>domain, and applying conditions on its distribution (which *you* can
>do yourself -- it's public domain, after all) is sort of pointless.
>
>Also, while I like the fact that RMS is writing neat stuff, I'm sort
>of annoyed with his idea that everyone is *obligated* to use Gnu-style
>licensing.  Let's give people the freedom to decide the conditions
>*they* want on distribution of their code.

I see your point.  Rather than requiring GNU-style licensing, I'll
just add it myself to code that contains no other copyright or
licensing agreement.  And I'll have to refuse to archive anything
that's not freely distributable.

Okay?

-Dave Sill
 dsill@relay.nswc.navy.mil
 elisp archive coordinator

wjc@ho5cad.ATT.COM (Bill Carpenter) (06/24/89)

In article <8906221943.AA10716@ucbvax.Berkeley.EDU> dsill@RELAY.NSWC.NAVY.MIL writes:

> One unresolved issue is how to name the archive files.  In most cases,
> I can simply use the author's name.  Sometimes, however, the author
> hasn't provided a name, or the name isn't unique, or the code consists
> of modifications to a distributed package such as rmail.

I reiterate my suggestion that authors provide 13-or-fewer-character
names (including ".el") for these files.  Although many users are on
long filename systems (including osu-cis), many others are in the land
of 14's.  It may seem a pain to make the sacrifice, but it seems
better than having to have half the population randomly renaming many
of the files they use from the archive.
--
   Bill Carpenter         att!ho5cad!wjc  or  attmail!bill

dsill@RELAY.NSWC.NAVY.MIL (06/24/89)

|From: sdr@flash.Alliant.COM (Stephen D. Rogers)
|
|[Directory names should be singular, e.g., "mode" vs "modes",
|"function" vs "functions", etc.]

That's fine with me.  That much less to type.

|I think that file names in the archive should be based on package names,
|prefixed by the login name of the author.  If login names are duplicated,
|add a digit to end of the login name.  The same prefix should be used for
|the same author throughout the archive.  For example,
|
|	(autoload 'C++-mode "mode/sdr-C++")

Not a bad idea.

|It would also be worthwhile to provide an AUTHORS file for the archive,
|which provided a cross reference of author id, name, and e-mail address.
|For example,
|
|	sdr	"Stephen D. Rogers"	srogers@graphics.alliant.com

Oops, I thought the author's ID would be his login name.  Otherwise, a
good idea.

|One last thought: If file name length is an issue, perhaps the author
|specification should be an additional directory level, rather than
|a prefix.  For example,
|
|	(autoload 'shell-mode "mode/sdr1/shell")

I suppose it is.  I'd like to keep all names at or below 14 characters
(13 for .el files to prevent conflict with the corresponding .elc
file.) for SysV folks.  Should I even try to indicate the type of the
file using an extension?  How about .el (elisp), .shar (shar archive),
.pat (diffs intended for patch)?

|Maybe better: we should just add the digit to the package name as
|an extension.  For example,
|
|	(autoload 'shell-mode "mode/shell.1")
|
|In this case, the first field in the AUTHORS file should be the
|relative package name instead of the author id.

You lost me here.  Could you explain that differently?

-Dave Sill
 dsill@relay.nswc.navy.mil
 elisp archive coordinator

sdr@flash.alliant.com (Stephen D. Rogers) (06/24/89)

>>From: sdr@flash.Alliant.COM (Stephen D. Rogers)
>From: dsill@relay.nswc.navy.mil

== More on the author id ==
>>It would also be worthwhile to provide an AUTHORS file for the archive,
>>which provided a cross reference of author id, name, and e-mail address.
>>For example,
>>
>>	sdr	"Stephen D. Rogers"	srogers@graphics.alliant.com
>
>Oops, I thought the author's ID would be his login name.  Otherwise, a
>good idea.

It would be, unless someone with that login name has already submitted
something to the archive.  Remember, in that case I suggested that we
tack a digit onto the login name.  For example, someone else with the
login name `sdr' might submit something to the archive, and would have
to be known as `sdr1'.  Their entry in the AUTHORS file might look
like this,

	sdr1	"Stefan D. Roland"	sdr@somewhere.com

I called the first field an author id because sometimes it isn't
the author's login name (because of the digit tacked on).

== Extensions and shar/patch pieces ==
>>One last thought: If file name length is an issue, perhaps the author
>>specification should be an additional directory level, rather than
>>a prefix.  For example,
>>
>>	(autoload 'shell-mode "mode/sdr1/shell")
>
>I suppose it is.  I'd like to keep all names at or below 14 characters
>(13 for .el files to prevent conflict with the corresponding .elc
>file.) for SysV folks.  Should I even try to indicate the type of the
>file using an extension?  How about .el (elisp), .shar (shar archive),
>.pat (diffs intended for patch)?

Some extensions I have seen in use: .el (elisp), .shar (shar archive),
.patch (diffs intended for patch), .Z (compress).

It seems like shar archives often come in numbered pieces, and patches
are numbered too.  So, you could put the pieces in a `shar' or `patch'
subdirectory.  Again, this makes it easier to live with restrictions on
file name length.  For example,

	mode/sdr1/shell/shar/01
	mode/sdr1/shell/shar/02
	...
	mode/sdr1/shell/shar/10

The leading zeroes make things sort properly.

Just as an aside, people without file name length restrictions
could easily transform these kinds of pathnames.  For example,

		     sdr1/shell ==> sdr1-shell
			shar/01 ==> 01.shar
yielding

	mode/sdr1/shell/shar/01 ==> mode/sdr1-shell/01.shar

The archive could easily provide scripts to do this.  Even better,
if the archive was maintained on a machine that didn't have file
name length restrictions, we could provide scripts to transform
the other way (to limited length names).

== If we didn't embed author id in package pathname... another way ==
>>Maybe better: we should just add the digit to the package name as
>>an extension.  For example,
>>
>>	(autoload 'shell-mode "mode/shell.1")
>>
>>In this case, the first field in the AUTHORS file should be the
>>relative package name instead of the author id.
>
>You lost me here.  Could you explain that differently?
>
>-Dave Sill
> dsill@relay.nswc.navy.mil
> elisp archive coordinator

Lets see...  I hope you don't mind, but I've thought of a slightly
better way, so I'll just describe that.

I would rather have the author id embedded in the package pathname, but
if it wasn't, then we would need to worry about package name collisions.
To resolve them, we could tack a number onto the end of the package
name.  For example,

	(autoload 'shell-1-mode "mode/shell-1")
	(autoload 'shell-2-mode "mode/shell-2")

sets up to load two different shell mode packages.

Then, we would need a PACKAGES file which provided a cross reference
between package names and the package's author.  The first field would
be the package name, and the second would be the author id.  For example,

	shell-1		sdr1

says that author `sdr1' wrote shell-1.  By referring to the AUTHORS file,
we know that sdr1 is "Stefan D. Roland", and so forth.

Again, I would rather have the author id embedded in the package pathname.

Stephen D. Rogers		srogers@graphics.alliant.com
Alliant Computer Systems	(508)486-4844
One Monarch Drive
Littleton, MA 01460

tale@pawl.rpi.edu (David C Lawrence) (06/24/89)

In <8906232104.AA01548@flash.> sdr@flash.alliant.com (Stephen D. Rogers) writes
sdr> It seems like shar archives often come in numbered pieces, and patches
sdr> are numbered too.  So, you could put the pieces in a `shar' or `patch'
sdr> subdirectory.  Again, this makes it easier to live with restrictions on
sdr> file name length.  For example,

sdr> 	mode/sdr1/shell/shar/01
sdr> 	mode/sdr1/shell/shar/02
sdr> 	...
sdr> 	mode/sdr1/shell/shar/10

AcK!  Deep directories in anonymous ftp hierarchies are a major pain
in the arse.  Furthermore, looking at an ls of mode, I am liable to
see something like "sdr  tale  dsill  kjones"  or some such -- those
aren't modes and they really make it more difficult for me to find
what I want because I either have to look it up in a table somewhere
or go looking through multiple ls' until I find the mode name I am
looking for.  For most packages (certainly >90%) there is no name
conflict at all.  For some, like pascal.el and my recently posted
irc.el, there is a conflict.  These can be resolved by _then_ adding
the author name to the package name or by making a (for example) irc
directory which would have three packages each identified by author
name.   And all the shar files should be together on the higher level,
nor down another path.  If I am looking at mode/perl and I see "shar",
the next thing I do is "get shar"; with the proposed "defense in
depth" method I get some annoying results because of trying to get a
directory.  

I was the one that first suggested to Dave that .el files have a
maximun name length of thirteen characters for SYSVers.  It is
certainly possible to do this without making the directories so
deep.

Dave
--
 (setq mail '("tale@pawl.rpi.edu" "tale@itsgw.rpi.edu" "tale@rpitsmts.bitnet"))
  "I realize the Internet isn't the whole world, but it is the center of it."
                                                        -- Greg Woods

kjones@talos.UUCP (Kyle Jones) (06/24/89)

Dale Worley writes:
 > Also, while I like the fact that RMS is writing neat stuff, I'm sort
 > of annoyed with his idea that everyone is *obligated* to use Gnu-style
 > licensing.  Let's give people the freedom to decide the conditions
 > *they* want on distribution of their code.

RMS has never insisted (at least not to me) that the GNU licensing be
used.  In fact saveconf.el (nee ultra.el) was distributed in v18.52 with
my own copying permissions.  The perennial Emacs favorite hanoi.el is
distributed with no copying permissions at all.

RMS does encourage use of the GNU General Public License, but the reason
he has always given me is that it makes things easier for the users to
have one set of copying terms.  I agree, so I use the license.

kjones@talos.UUCP (Kyle Jones) (06/24/89)

Stephen D. Rogers writes:
 > I think that the same directory structure should be used for *archiving*
 > and for *using* lisp packages.  However, I also think that the directory
 > structure should be oriented towards package use first, and package
 > archiving second.

I disagree.  We're discussing an archive, not a library.  Organization
and file naming should be aimed toward easing the task of finding the
software you want and unpacking it once you have it on your system.
Once the code is on your system, you can organize it as you see fit.

The subdirectory scheme Dave Sill outline is quite sufficient.  I don't
see the need for subdirectories for individual packages.  Multifile
packages should be shar'd and compressed anyway.

As for file names, as long as the file name can be matched to a code
description in Dave Sill's elisp-directory, the file names can be jsut
about anything.  Shar files should be numbered appropriately, and have
"shar" in the file name somewhere.  Also I think all the code should be
compressed, for the sake of OSU's disks and to cut file transfer times.

rbr4@uhura.cc.rochester.edu (Roland Roberts) (06/27/89)

In article <8906221943.AA10716@ucbvax.Berkeley.EDU> dsill@RELAY.NSWC.NAVY.MIL
writes:
>Other subdirectories will be set up such as those suggested recently
>by Chris Siebenmann:
[elided]
>Any others?

termcap - I started working on a termcap for our Human Design Series
terminals but never completed it (I wimped out and used its vt220 emulation
mode).

-- 
Roland Roberts                      BITNET: roberts@uornsrl
  Nuclear Structure Research Lab  INTERNET: rbr4@uhura.cc.rochester.edu
  271 East River Road                 UUCP: rochester!ur-cc!uhura!rbr4
  Rochester, NY  14267                AT&T: (716) 275-8962