[comp.sys.xerox] 'provide' and 'require' and compiled files

uda@mina.liu.se (Ulf Dahlen) (01/27/89)

How is 'provide' and 'require' implemented in existing CL-implementations?
I would like 'require' to search for the compiled file first, and load it if
it's found. Xerox Lisp has a 'require' that doesn't bother about compiled
files. This makes it almost necessary to use the InterLISP function 'FILESLOAD'
and that will make programs less portable.

Let's say I have a file submodule that does the following:

(provide 'submodule)
;;; rest of file

Then say I compile this file, getting a submodule.fasl (or whatever extension
is used). If I have another file with these statements:

(provide 'mainmodule)
(require 'submodule)
;;; rest of file

I would like this file to load submodule.fasl, not submodule.lisp. The fact
that the extension for compiled files is not stated in CLtL, makes it
impossible to write (require 'submodule 'submodule.fasl) or something like
that.

__________
Ulf Dahlen
Dept of Computer & Info Science, University of Linkoping, Sweden
Troskaregatan 51:23       |     uda@ida.liu.se
S-583 30  LINKOPING       |     uda@majestix.liu.se, uda@majestix.UUCP
SWEDEN                    |     {mcvax,munnari,seismo}!enea!liuida!uda
"The beginning is a very delicate time."

cutting.pa@XEROX.COM (01/31/89)

"I would like 'require' to search for the compiled file first, and load it
if
it's found. Xerox Lisp has a 'require' that doesn't bother about compiled
files. This makes it almost necessary to use the InterLISP function
'FILESLOAD'
and that will make programs less portable."

You must be using Lyric.  In Medley 'require' behaves more like
IL:FILESLOAD.  It searches the connected directory and directories named on
IL:DIRECTORIES, first for compiled files then for source files.

	Doug

masinter.pa@XEROX.COM (02/01/89)

Just FYI, it is likely that the draft standard for ANSI Common
Lisp will omit REQUIRE and PROVIDE even though very few
other functions from CLtL are to be dropped. 

This isn't absolute final word -- that won't come until after there 
is a draft standard and then after it gets approved -- but you can 
see that the answer to the question "How is 'provide' and 'require' 
implemented in existing CL-implementations?" is "inconsistently."


Issue:         REQUIRE-PATHNAME-DEFAULTS
References:    *MODULES*, PROVIDE, REQUIRE, pp 188-191
               LOAD, pp 426-427
Category:      CHANGE
Edit history:  Version 1 by Pierson 9/13/88
               Version 2 by Pierson 9/19/88, change PROVIDE stuff per comments
               Version 3 by Pierson 10/17/88, remove PROVIDE locaction specs.
               Version 4 by Pierson 10/31/88, remove from language
               Version 5 by Pierson 11/15/88, cleanup, fix discussion
               Version 6 by Pierson 12/9/88, remove *MODULES* as well

Problem description:

PROVIDE and REQUIRE are a dual-purpose pair of functions that attempt
to provide multi-file Common Lisp programs with a single mechanism to
detect and correct incorrect load sequences.  These functions were
also designed to be used for general file inclusion in Common Lisp.
Unfortunately, the file loading feature of REQUIRE is specified such
that it is inherently non-portable and environment dependent.

Proposal (REQUIRE-PATHNAME-DEFAULTS:ELIMINATE):

Remove PROVIDE, REQUIRE, and *MODULES* from the Common Lisp standard.

Test Cases/Examples:

(PROVIDE 'fft)

Would not be Common Lisp.

(REQUIRE 'fft)

Would not be Common Lisp.

Rationale:

The file loading feature of REQUIRE is non-portable.  The remaining
functionality of PROVIDE and REQUIRE (pushing and testing *MODULES*)
can easily be implemented by user code.  Since some implementations
will retain the automatic module loading features of REQUIRE and some
won't, use of REQUIRE will almost always make code less portable.

Current practice:

All implementations currently support some sort of file loading via
single-argument REQUIRE.  In general, the Lisp Machine implementations
invoke the system module building/loading facility while the Unix
implementations simply try to load a file in the current directory.

Cost to Implementors:

Implementations will have to move PROVIDE and REQUIRE to their package
for implementation extensions and change their documentation to
indicate that PROVIDE and REQUIRE are non-standard.  This is a fairly
small change.

Cost to Users:

Non-portable programs that rely on PROVIDE and REQUIRE will probably
be unaffected since implementations will probably maintain their
existing functionality.  Since the current behavior is decidedly
non-portable, portable programs have to aviod or special-case PROVIDE
and REQUIRE anyway.

Cost of non-Adoption:

PROVIDE and REQUIRE will continue as impediments to portability.

Benefits:

The non-portability of PROVIDE and REQUIRE will be made obvious.

Aesthetics:

This simplifies the language by removing an environment-dependent
feature. 

Discussion:

The cleanup committee tried to come up with a proposal to restrict
PROVIDE and REQUIRE to the portable subset of their functionality.
This failed because several implementors objected that it compelled
them to significantly reduce the functionality they provided users in
order to create a trivial feature which any user could easily write
for herself.

Fahlman, Gregor, Grey, Loosemore, Moon, Pierson, Pitman, Steele, and
Zacharias have expressed support for removing PROVIDE and REQUIRE from
the language, at least as the lesser of several evils.

JonL would much rather see PROVIDE and REQUIRE remain in the language
as a safety net behind any implementation-specific system building
facility.  Pierson likes the safety net idea, but doesn't think it's
workable without forbidding REQUIRE from loading files.

Pitman suggested that PROVIDE and REQUIRE should be depricated rather
than removed entirely.  Pierson agrees, but notes that Larry wants us
to deal with deprication versus elimination as a separate global topic.

Several people have expressed a desire not to break existing user
code.  If accepted, this proposal should not break existing code
because all implementations are expected to retain their current
PROVIDE and REQUIRE functionality as an extension to Common Lisp.

andy@Gang-of-Four.Stanford.EDU (Andy Freeman) (02/01/89)

In article <1168@mina.liu.se> uda@mina.liu.se (Ulf Dahlen) writes:
>How is 'provide' and 'require' implemented in existing CL-implementations?
>I would like 'require' to search for the compiled file first, and load it if
>it's found. Xerox Lisp has a 'require' that doesn't bother about compiled
>files. This makes it almost necessary to use the InterLISP function 'FILESLOAD'
>and that will make programs less portable.

Given this description, I can't tell where the problem is.  Steele pg
188 says:

"The require function tests whether a module is already present (using
a case-sensitive comparison); if the module is not present, require
proceeds to load the appropriate file or set of files.  The pathname
argument, if present, is a single pathname or a list of pathnames
whose files are to be loaded in order, left to right.  If the pathname
argument is nil or is not provided, the system will attempt to
determine in some system-dependent manner, which files to load.  This
will typically involve some central registry of module names and the
associated file lists."

Note that require's second argument is a pathname, not a string or a
symbol.  The string/symbol "representation" of a pathname on one
implementation may not work on another implementation, but the
pathname construction functions must.

As to getting require to do the right thing if there are both compiled
and text versions of the file, I refer readers to the description of
load, on page 426.  (The description of require implies that it uses
load.)

"If the filename (after the merging in of the defaults does not
explicitly specify a type, and both text and object types of the file
are available in the file system, load should try to select the more
appropriate file by some implementation-dependent means."

This is why it is unnecessary (in this case) for CL to specify the
file type for compiled files.  (It would be nice if CL defined
pathname defaults constants whose implementation dependent values had
the various file types, directories, etc in them, but they're not
needed in this case because load does the right thing.)

In other words, don't do the following:

>(require 'submodule 'submodule.fasl)

as (require 'submodule <expression whose value is a pathname for submodule
that doesn't specify the file type>) does the right thing and is portable.

Did the ANSI people really eliminate require and provide (or require's
second argument) because they couldn't figure this out, or was there
some other reason?  (If ANSI CL doesn't have pathname functions, or
ANSI found a hole in CL's, then that may explain it.  However, it
would be better for ANSI to include pathname functions or fix the
hole.)

-andy
UUCP:  {arpa gateways, decwrl, uunet, rutgers}!polya.stanford.edu!andy
ARPA:  andy@polya.stanford.edu
(415) 329-1718/723-3088 home/cubicle

barmar@think.COM (Barry Margolin) (02/01/89)

In article <6508@polya.Stanford.EDU> andy@Gang-of-Four.Stanford.EDU (Andy Freeman) writes:
>Did the ANSI people really eliminate require and provide (or require's
>second argument) because they couldn't figure this out, or was there
>some other reason?  (If ANSI CL doesn't have pathname functions, or
>ANSI found a hole in CL's, then that may explain it.  However, it
>would be better for ANSI to include pathname functions or fix the
>hole.)

Below is the full text of the proposal that we passed two weeks ago.
Basically, we realized that any use of the second argument to REQUIRE
is non-portable; any full pathname is obviously specific to a
particular file system organization, and an incomplete pathname
depends upon some implementation-specific searching mechanism.  And
the use of the first argument to REQUIRE is also dependent upon
implementations.

ANSI CL will have all the same pathname functions as CLtL describes.
However, if you've ever tried to port a program that uses them between
Lucid CL and Symbolics CL, both of which mostly conform to CLtL, you
will have learned that they are extremely underspecified.

Issue:         REQUIRE-PATHNAME-DEFAULTS
References:    *MODULES*, PROVIDE, REQUIRE, pp 188-191
               LOAD, pp 426-427
Category:      CHANGE
Edit history:  Version 1 by Pierson 9/13/88
               Version 2 by Pierson 9/19/88, change PROVIDE stuff per comments
               Version 3 by Pierson 10/17/88, remove PROVIDE locaction specs.
               Version 4 by Pierson 10/31/88, remove from language
               Version 5 by Pierson 11/15/88, cleanup, fix discussion
               Version 6 by Pierson 12/9/88, remove *MODULES* as well

Problem description:

PROVIDE and REQUIRE are a dual-purpose pair of functions that attempt
to provide multi-file Common Lisp programs with a single mechanism to
detect and correct incorrect load sequences.  These functions were
also designed to be used for general file inclusion in Common Lisp.
Unfortunately, the file loading feature of REQUIRE is specified such
that it is inherently non-portable and environment dependent.

Proposal (REQUIRE-PATHNAME-DEFAULTS:ELIMINATE):

Remove PROVIDE, REQUIRE, and *MODULES* from the Common Lisp standard.

Test Cases/Examples:

(PROVIDE 'fft)

Would not be Common Lisp.

(REQUIRE 'fft)

Would not be Common Lisp.

Rationale:

The file loading feature of REQUIRE is non-portable.  The remaining
functionality of PROVIDE and REQUIRE (pushing and testing *MODULES*)
can easily be implemented by user code.  Since some implementations
will retain the automatic module loading features of REQUIRE and some
won't, use of REQUIRE will almost always make code less portable.

Current practice:

All implementations currently support some sort of file loading via
single-argument REQUIRE.  In general, the Lisp Machine implementations
invoke the system module building/loading facility while the Unix
implementations simply try to load a file in the current directory.

Cost to Implementors:

Implementations will have to move PROVIDE and REQUIRE to their package
for implementation extensions and change their documentation to
indicate that PROVIDE and REQUIRE are non-standard.  This is a fairly
small change.

Cost to Users:

Non-portable programs that rely on PROVIDE and REQUIRE will probably
be unaffected since implementations will probably maintain their
existing functionality.  Since the current behavior is decidedly
non-portable, portable programs have to aviod or special-case PROVIDE
and REQUIRE anyway.

Cost of non-Adoption:

PROVIDE and REQUIRE will continue as impediments to portability.

Benefits:

The non-portability of PROVIDE and REQUIRE will be made obvious.

Aesthetics:

This simplifies the language by removing an environment-dependent
feature. 

Discussion:

The cleanup committee tried to come up with a proposal to restrict
PROVIDE and REQUIRE to the portable subset of their functionality.
This failed because several implementors objected that it compelled
them to significantly reduce the functionality they provided users in
order to create a trivial feature which any user could easily write
for herself.

Fahlman, Gregor, Grey, Loosemore, Moon, Pierson, Pitman, Steele, and
Zacharias have expressed support for removing PROVIDE and REQUIRE from
the language, at least as the lesser of several evils.

JonL would much rather see PROVIDE and REQUIRE remain in the language
as a safety net behind any implementation-specific system building
facility.  Pierson likes the safety net idea, but doesn't think it's
workable without forbidding REQUIRE from loading files.

Pitman suggested that PROVIDE and REQUIRE should be depricated rather
than removed entirely.  Pierson agrees, but notes that Larry wants us
to deal with deprication versus elimination as a separate global topic.

Several people have expressed a desire not to break existing user
code.  If accepted, this proposal should not break existing code
because all implementations are expected to retain their current
PROVIDE and REQUIRE functionality as an extension to Common Lisp.

Barry Margolin
Thinking Machines Corp.

barmar@think.com
{uunet,harvard}!think!barmar