scott@wiley.uucp (Scott Simpson) (10/17/90)
I posted a message before complaining about the Common Lisp package system and I received some messages that still left me with this vague empty feeling. I have been having trouble accepting the way Common Lisp handles the relationships between object-oriented programming in CLOS and information hiding (which in Common Lisp is chiefly handled by packages). My first suggestion was to make a separate package for each class. My reasons for this were simple: I wished to hide everything except what I explicitly wished to export behind the class wall and only give the user of my class access to to the functions I dictate. This is the essence of data abstraction anyhow. Any support functions that were needed for the functions I exported could not be seen behind the class (i.e., package) wall. Clients of my class could refer to my exported functions as (window:move ...) for example. I like the use of the package prefix and virtually *never* advocate the use of use-package. Likewise, when I used Ada, I virtually never used the "use" clause which is similar to Lisps. Many Ada gurus agree with me. The reason is that with "use" you overpopulate the name space with easily accessible garbage and you cannot find where an entity comes from. If you leave the class prefix on, then you can immediately see where a class comes from. (Lisp allows you to selectively import certain names which is much safer though. Also, a hypertext like browser could alleviate the entity origin problem.) I also liked Lisps ability to conform to Meyer's Principle of Uniform Reference (see "Object-Oriented Software Construction", Bertrand Meyer, a must read) which states that you shouldn't know whether you get a value by stored state or computation. All looked wonderful in Lispland. It was not to be. With each class a separate package, you must painfully create an export clause for each entity exported. While I have no objection to this (for it is done is languages like Eiffel and it is quite nice), I do start to complain when functions such as defclass create a whole bunch of methods behind your back that you wish to have exported. You have to know which methods defclass is going to create and manually export them. This could be done by redefining defclass at the metaclass level but this is a kludge. Also, inheritance becomes a problem because you must not only export all the functions of your current class but also all the functionality you wish exported from your superclasses. There is something disconcerting to me about looking up the inheritance tree to see all the functions that you are inheriting. (C++ gives great control over inheritance control. Perhaps too much. I'm not alone here.) Things were going from bad to worse so I flipped the coin and did what everybody else was advising me; I got rid of packages all together. This had some nice effects. First, I no longer had all those painful export clauses. I also didn't have to look up the tree to see what methods my parents had and export those too. Unfortunately, I also didn't have information hiding. Any support functions that I needed and didn't wish to export were visible. Any variables that I wished local to my class weren't. And worst of all, I started to run into massive amounts of name conflicts. I'm not talking about code created by multiple users; I'm talking about name conflicts between routines I wrote myself. With the package approach, it was nice to be able to call routines with the same name but in different packages such as (unix:lock ...) and (semaphore:lock ...). The colon had a nice "class wall" kind of feel to it. Now I had to say unix-lock and semaphore-lock. I could see myself deevolutionizing to my old emaciated C approach. The dash didn't feel like a class wall; it looked like millions of other kludgy identifiers I had seen before. I had adopted a naming convention because the language didn't give me the power I wanted. CLOS wasn't of any help. If I defined a reader function, I had to give the full name of the reader function rather than defaulting to the class name followed by the slot name (e.g., ":reader lock-name" for the lock class). If I forgot to put the class name as a prefix in some of the :reader clauses ---> symbol conflict! Worse yet, the Lisp system we were using (Mercury from Artificial Intelligence Technologies; an almost CLOS) defaulted to *not* putting the prefix on an accessor method, so any two slots withing the same name in the same package generated a symbol conflict! I have generated a table of the information I have culled from my experimentation with Common Lisp: | with package per class | many classes per package ------|--------------------------------|----------------------------------- |Class prefixes begin with a |Do not manually need to export pro |colon. |superclass methods. |Good information hiding is | |achieved. | --------------------------------------------------------------------------- |Must manually export superclass |You end up with symbol names that con |methods. |have kludgy dash prefixes. |Must manually export methods |You don't have good information |created by defclass. |hiding. --------------------------------------------------------------------------- As an aside on use-package. If you do use it, you know that you must shadow any symbols in the used package if they already exist in the current package. For example, if you have a symbol CLASS in the current package, then you will have to do (shadow 'class) (use-package 'MOS) Now you will refer to USER:CLASS and not MOS:CLASS. I don't like these semantics because you now have two ways to refer to two different symbols with the same name. In the USER case, you can just refer to it as CLASS. In the MOS case, you have to refer to it as MOS:CLASS. I like Ada semantics better. In Ada, you can refer to any symbols used by just using their name unless there is a conflict. If there is a conflict, you must qualify *both* of them. This seems like a more consistent treatment. I am guessing that Lisp does not use this approach because it would have to search the symbol tables of all packages whenever it encountered a symbol in the current package to see if there was a conflict. This would be an unacceptable performance penalty.
pab@lucid.com (Peter Benson) (10/17/90)
In article <271BA6D1.5B83@wilbur.coyote.trw.com> scott@wiley.uucp (Scott Simpson) writes:
I posted a message before complaining about the Common Lisp package
system and I received some messages that still left me with this vague
empty feeling.
The CL package system is pretty hard for most people to deal with and
understand.
. . .
My first suggestion was to make a separate package for each
class. My reasons for this were simple: I wished to hide everything
except what I explicitly wished to export behind the class wall and
only give the user of my class access to to the functions I dictate.
This seems like overkill. It is so easy to get confused with packages that
fewer packages is usually better. A better way to go is to have a
different package for each large conceptual block of your program. You
don't want to have zillions of exports, so you want to modularize well and
then export the (hopefully not too large) interface to each module.
. . .
It was not to be. With each class a separate package, you must
painfully create an export clause for each entity exported. While I
have no objection to this (for it is done is languages like Eiffel and
it is quite nice), I do start to complain when functions such as
defclass create a whole bunch of methods behind your back that you
wish to have exported. You have to know which methods defclass is
going to create and manually export them.
There shouldn't be any method defined behind your back. All the accessors,
readers, and writers must be put in the slot definition explicitly. No make
methods are created. Make-instance handles it all. I don't know anything
about the clos implementation you are using, but you want to try to either
ignore or suppress the automatically created methods and only use the
explicit forms. That way you can port it to another clos implementation.
You can get your hiding done better by writing another macro that expands
into a defclass and some exports.
This could be done by
redefining defclass at the metaclass level but this is a kludge.
Stay away from that if possible. The meta object protocol is still going
through some flux.
Also, inheritance becomes a problem because you must not only
export all the functions of your current class but also all the
functionality you wish exported from your superclasses. There is
something disconcerting to me about looking up the inheritance tree to
see all the functions that you are inheriting.
It can't really work because packages only give one level of "inheritance".
You would always end up looking at the class hierarchy, which is a bad
thing to have to do.
. . .
Things were going from bad to worse so I flipped the coin and
did what everybody else was advising me; I got rid of packages all
together. This had some nice effects. First, I no longer had all those
painful export clauses. I also didn't have to look up the tree to see
what methods my parents had and export those too.
Now you are going too far in the other direction. You want a few packages
and well defined interfaces that you can export without pain.
Unfortunately, I also didn't have information hiding. Any
support functions that I needed and didn't wish to export were
visible. Any variables that I wished local to my class weren't. And
worst of all, I started to run into massive amounts of name conflicts.
I'm not talking about code created by multiple users; I'm talking
about name conflicts between routines I wrote myself. With the package
approach, it was nice to be able to call routines with the same name
but in different packages such as (unix:lock ...) and (semaphore:lock
...). The colon had a nice "class wall" kind of feel to it. Now I had
to say unix-lock and semaphore-lock.
Wait a minute. This is what generic functions are for. The generic
function LOCK should work on unix lock instances and semaphore instances
without any conflict. Also the accessors defined by defclass should be
generic functions, and having a same named accessor for to completely
different classes should be at worst a performance hit.
. . .
language didn't give me the power I wanted. CLOS wasn't of any help.
If I defined a reader function, I had to give the full name of the
reader function rather than defaulting to the class name followed by
the slot name (e.g., ":reader lock-name" for the lock class). If I
forgot to put the class name as a prefix in some of the :reader
clauses ---> symbol conflict! Worse yet, the Lisp system we were using
(Mercury from Artificial Intelligence Technologies; an almost CLOS)
defaulted to *not* putting the prefix on an accessor method, so any
two slots withing the same name in the same package generated a symbol
conflict!
The accessor names must be spelled out completely for a reason. You said
you didn't want defclass making methods behind your back. If you want
automatically created accessors or aaccessor names then write another
macro.
. . .
As an aside on use-package. If you do use it, you know that
you must shadow any symbols in the used package if they already exist
in the current package. For example, if you have a symbol CLASS in
the current package, then you will have to do
(shadow 'class)
(use-package 'MOS)
Now you will refer to USER:CLASS and not MOS:CLASS. I don't like these
semantics because you now have two ways to refer to two different
symbols with the same name. In the USER case, you can just refer to it
as CLASS. In the MOS case, you have to refer to it as MOS:CLASS. I
like Ada semantics better. In Ada, you can refer to any symbols used
by just using their name unless there is a conflict. If there is a
conflict, you must qualify *both* of them. This seems like a more
consistent treatment. I am guessing that Lisp does not use this
approach because it would have to search the symbol tables of all
packages whenever it encountered a symbol in the current package to
see if there was a conflict. This would be an unacceptable performance
penalty.
That's not a bad idea. It's not quite shadowing. You wouldn't believe the
contortions that the reader and loader have to go through anyway for
packages. I don't think this would be that significant of a performance hit.
The problem comes in with compiled files that have symbols in them. There
is usually no way to tell, once a symbol is read, whether or not it was
package qualified. So a compiled file may have a reference to a symbol and
have no information on whether or not it was qualified. Don't even think
about what happens when the package system is different at the time it is
loaded from when it was compiled.
-ptr-
pab@lucid.comscott@wiley.uucp (Scott Simpson) (10/18/90)
In article <2298@heavens-gate.lucid.com> pab@lucid.com (Peter Benson) writes: >In article <271BA6D1.5B83@wilbur.coyote.trw.com> scott@wiley.uucp (Scott Simpson) writes: > My first suggestion was to make a separate package for each > class. My reasons for this were simple: I wished to hide everything > except what I explicitly wished to export behind the class wall and > only give the user of my class access to to the functions I dictate. > >This seems like overkill. It is so easy to get confused with packages that >fewer packages is usually better. A better way to go is to have a >different package for each large conceptual block of your program. You >don't want to have zillions of exports, so you want to modularize well and >then export the (hopefully not too large) interface to each module. Since I haven't quite figured out how packages can be used with CLOS classes usefully I can't make a judgement on their power and whether what I am doing is overkill or not. I am not convinced that packages even deserved to be given the connotation "powerful". So far I have found them not to be. I do think it is useful though to be able to export some routines from a class and not export other routines. These other non-exported routines are used to support the exported routines but have no business being exported because they don't support the data abstraction. I agree with you that packages in Common Lisp do not work well as my class firewall. There are too many idiosyncracies that go along with them. Suppose I don't use them then and suppose I have two classes A and B and suppose a function X in class A is a support routine for one of the methods of A. Then X should not be exported from A. Also suppose class B is defined in the same package as A. Then the routines of class B can also get to X when they shouldn't be able to. As for manually listing all the routines you wish to export to the outside world, I say why not! How else are you going to specify what behavior should be seen by clients of the class and what behavior is strictly there for implementation purposes. Eiffel forces you to do this. > It was not to be. With each class a separate package, you must > painfully create an export clause for each entity exported. While I > have no objection to this (for it is done is languages like Eiffel and > it is quite nice), I do start to complain when functions such as > defclass create a whole bunch of methods behind your back that you > wish to have exported. You have to know which methods defclass is > going to create and manually export them. > >There shouldn't be any method defined behind your back. All the accessors, >readers, and writers must be put in the slot definition explicitly. No make >methods are created. Make-instance handles it all. I don't know anything >about the clos implementation you are using, but you want to try to either >ignore or suppress the automatically created methods and only use the >explicit forms. That way you can port it to another clos implementation. >You can get your hiding done better by writing another macro that expands >into a defclass and some exports. I screwed up on this one. The pseudo-implementation of CLOS I am using (Mercury) creates a predicate LOCK-P if you create a class LOCK. It also creates an initialization generic called INIT-LOCK that you can assign before and after methods to. Both of these have to be manually exported although I don't create them. CLOS uses TYPEP rather than creating a predicate LOCK-P. However, both Mercury and CLOS must manually export the symbol LOCK or other packages will not be able to say (typep *lock* 'lock:lock). Also, if packages are used as class firewalls, then it would be much easier to have the metaclass create an export clause for each reader and writer automatically. > Things were going from bad to worse so I flipped the coin and > did what everybody else was advising me; I got rid of packages all > together. This had some nice effects. First, I no longer had all those > painful export clauses. I also didn't have to look up the tree to see > what methods my parents had and export those too. > >Now you are going too far in the other direction. You want a few packages >and well defined interfaces that you can export without pain. And if I have multiple classes within one package then I have information hiding problems. > Unfortunately, I also didn't have information hiding. Any > support functions that I needed and didn't wish to export were > visible. Any variables that I wished local to my class weren't. And > worst of all, I started to run into massive amounts of name conflicts. > I'm not talking about code created by multiple users; I'm talking > about name conflicts between routines I wrote myself. With the package > approach, it was nice to be able to call routines with the same name > but in different packages such as (unix:lock ...) and (semaphore:lock > ...). The colon had a nice "class wall" kind of feel to it. Now I had > to say unix-lock and semaphore-lock. > >Wait a minute. This is what generic functions are for. The generic >function LOCK should work on unix lock instances and semaphore instances >without any conflict. Also the accessors defined by defclass should be >generic functions, and having a same named accessor for to completely >different classes should be at worst a performance hit. Yes, generic functions work. Sorry, I should have thought of them. I still like the package prefix though for the reasons I outlined in my previous message. Again, a hypertext like interface would wean me of this preference. > language didn't give me the power I wanted. CLOS wasn't of any help. > If I defined a reader function, I had to give the full name of the > reader function rather than defaulting to the class name followed by > the slot name (e.g., ":reader lock-name" for the lock class). If I > forgot to put the class name as a prefix in some of the :reader > clauses ---> symbol conflict! Worse yet, the Lisp system we were using > (Mercury from Artificial Intelligence Technologies; an almost CLOS) > defaulted to *not* putting the prefix on an accessor method, so any > two slots withing the same name in the same package generated a symbol > conflict! > >The accessor names must be spelled out completely for a reason. You said >you didn't want defclass making methods behind your back. If you want >automatically created accessors or aaccessor names then write another >macro. I don't want automatically created accessors but I do want automatically created accessor names. If accessor names are not to have the package prefix, then I am torn between prefixing the slot name with the class name or not. I don't know which route to pick although <class-name>-<slot-name> currently seems to be the most popular route. (I'm looking at lexical closures too. Perhaps they will be my salvation.) Scott Simpson TRW scott@coyote.trw.com
miller@cam.nist.gov (Bruce R. Miller) (10/18/90)
In article <271CDC30.1E54@wilbur.coyote.trw.com>, Scott Simpson writes: > In article <2298@heavens-gate.lucid.com> pab@lucid.com (Peter Benson) writes: > >In article <271BA6D1.5B83@wilbur.coyote.trw.com> scott@wiley.uucp (Scott Simpson) writes: > > My first suggestion was to make a separate package for each > > class. My reasons for this were simple: I wished to hide everything... > > > >This seems like overkill. ... > > Since I haven't quite figured out how packages can be used with CLOS > classes usefully I can't make a judgement on their power and whether > what I am doing is overkill or not. I am not convinced that packages > even deserved to be given the connotation "powerful". ... (with-disclaimers-and-IMHO () I think that CLOS & Packages are two mechanisms with different scales and intents. And although they both seem to deal with `information hiding' they deal with it in quite different ways. One point is that, in Lisp, nothing is REALLY hidden -- some things are just a little less visible than others. Want a non-exported symbol from a package? Use two colons. Want to get at an unapproved slot from a clos instance? Just use slot-value. On the one hand, I think that packages are (very) useful at a coarse grain: for example at the application level. I've never written an application with more than 2 packages and even those were cases where the one might be used as a separate set of tools. [I do often write `user-level' programs for using an application and define a package for each `user program' which uses the application's package. It eases using the application's tools and avoids clashes between different, similar programs] They really come in useful when you have a well `lived in' lisp environment where there are multiple applications loaded and ready to run (or are running, if you have multi-tasking lisp environment). Of course, you determine which symbols should be used by outsiders by exporting them. If an outsider has to use two colons, he implicitly acknowledges the risk. Anyway, that's how I use packages, but maybe I'm an unsophisticated package user. On the other hand, it is my impression that with CLOS (I haven't done much CLOS yet, but I have done a lot with Flavors --- I think it's the same as far as this discussion is concerned) one differentiates between the `external' and `internal' interface to a class of objects simply by saying so! That is, the developer announces that these are the routines to use. Any other routines are for internal use only. Packages help in so far as it helps (loosely) enforce that distinction to users outside of the application, but doesn't help between modules within the same package. You just dont call the internals by virtue of your strength of will and extraordinary discipline! As you've seen, using too many packages within the same program gets hairy. But, I dont think that CLOS really attempts to enforce any kind of barrier, and packages seems to be at the wrong level for what you want --- lisp itself isn't really into firewalls in any case. A trick I've used with some satisfaction -- as a reminder, NOT a firewall -- is to document, literally, the `documented interface' by giving a documentation string (where appropriate). If I'm in doubt about invoking a function in a particular piece of code, If m-sh-D gives me documentation, I figure it's OK, otherwise probably not. --- OK, it's far from perfect, just an idea. bruce
scott@wiley.uucp (Scott Simpson) (10/19/90)
In article <2865200060@ARTEMIS.cam.nist.gov> miller@cam.nist.gov writes: >I think that CLOS & Packages are two mechanisms with different scales >and intents. And although they both seem to deal with `information >hiding' they deal with it in quite different ways. I agree that the intent that I intended to use packages for and the intent they were designed for are probably quite different. I don't know what packages were designed for. I have been told that they are to be used for separating large subsystems of code. The point I have been trying to make is that the information hiding granularity of packages is way too coarse and no other mechanism in Lisp can give me the granularity I want (i.e., class based) easily. I don't find packages very useful and I wouldn't cry a tear if they were simply removed from the language. I don't know if I agree that CLOS deals with information hiding. How do you mean? >One point is that, in Lisp, nothing is REALLY hidden -- some things are >just a little less visible than others. Want a non-exported symbol from >a package? Use two colons. Want to get at an unapproved slot from a >clos instance? Just use slot-value. Yes. I hate this. The double colon is way too similar to the single colon. I can concede that for efficiency reasons or some such it is non unreasonable to have back doors that can break the abstraction barrier. C++ does this with its ugly friend functions. However, I do think that they should be discouraged and if used, should be quite noticeable. It is said that they put gotos in Ada (especially for automatically generated Ada programs) but that they discourage their use and make them stick out like a sore thumb. Ada's labels look like <<label>>. >On the one hand, I think that packages are (very) useful at a coarse >grain: for example at the application level. I've never written an >application with more than 2 packages and even those were cases where >the one might be used as a separate set of tools. [I do often write If you hardly ever use packages, why do you find them so useful? >On the other hand, it is my impression that with CLOS (I haven't done >much CLOS yet, but I have done a lot with Flavors --- I think it's the >same as far as this discussion is concerned) one differentiates between >the `external' and `internal' interface to a class of objects simply by >saying so! That is, the developer announces that these are the routines >to use. Any other routines are for internal use only. Packages help in >so far as it helps (loosely) enforce that distinction to users outside >of the application, but doesn't help between modules within the same >package. You just dont call the internals by virtue of your strength of >will and extraordinary discipline! Exactly. Strength of will and extraordinary discipline. You hit the nail on the head. This is *exactly* what I am trying to avoid. Besides, if we didn't cherish information hiding, language protection and ease of uses why don't we all go back to programming in assembly language? They all are Turing machine equivalent anyway. Right? I don't think that the difference between external and internal functions should just be that one is documented and the other isn't. We need language features to help in providing protection and ease of use. >As you've seen, using too many packages within the same program gets >hairy. But, I dont think that CLOS really attempts to enforce any kind >of barrier, and packages seems to be at the wrong level for what you >want --- lisp itself isn't really into firewalls in any case. OK. Here you say that CLOS doesn't enforce any barrier. I agree. Also you are right. Lisp isn't into firewalls. This is my complaint. >A trick I've used with some satisfaction -- as a reminder, NOT a >firewall -- is to document, literally, the `documented interface' by >giving a documentation string (where appropriate). If I'm in doubt >about invoking a function in a particular piece of code, If m-sh-D gives >me documentation, I figure it's OK, otherwise probably not. --- OK, >it's far from perfect, just an idea. I think most Lisp users suffer in the same way. Scott Simpson TRW scott@coyote.trw.com
moore%cdr.utah.edu@cs.utah.edu (Tim Moore) (10/19/90)
In article <271E0D40.451E@wilbur.coyote.trw.com> scott@wiley.UUCP (Scott Simpson) writes: >In article <2865200060@ARTEMIS.cam.nist.gov> miller@cam.nist.gov writes: > >>One point is that, in Lisp, nothing is REALLY hidden -- some things are >>just a little less visible than others. Want a non-exported symbol from >>a package? Use two colons. Want to get at an unapproved slot from a >>clos instance? Just use slot-value. > >Yes. I hate this. The double colon is way too similar to the single >colon. I can concede that for efficiency reasons or some such it is >non unreasonable to have back doors that can break the abstraction >barrier. C++ does this with its ugly friend functions. However, I do >think that they should be discouraged and if used, should be quite >noticeable. It is said that they put gotos in Ada (especially for >automatically generated Ada programs) but that they discourage their >use and make them stick out like a sore thumb. Ada's labels look like ><<label>>. I don't see what the problem is. If you don't want to get burned by using internal functions and data structures, don't use them. I think it's great that data structures are exposed in Lisp, if only to satisfy my own curiosity. One of the first things I do when I use a Common Lisp implementation for the first time is type (symbol-plist '+) to see what turns up. (Maybe that comes after (time (fact 1000)) :-) Some Lisp implementations go to great trouble to insure that the source code of any function is available for inspection. That is even better. >>On the one hand, I think that packages are (very) useful at a coarse >>grain: for example at the application level. I've never written an >>application with more than 2 packages and even those were cases where >>the one might be used as a separate set of tools. [I do often write > >If you hardly ever use packages, why do you find them so useful? That's not what he said at all. The basic idea behind packages is that it's not too hard to avoid name collisions in your own code, or even within an "application" (you can use grep, after all), but it is hard to avoid collisions with other random programs that might be loaded in the same Lisp image. For one program, one or two packages is about right. >>[The distinction between "internal" and "external" is in the documentation] >>... >>of the application, but doesn't help between modules within the same >>package. You just dont call the internals by virtue of your strength of >>will and extraordinary discipline! > >Exactly. Strength of will and extraordinary discipline. It doesn't take extraordinary discipline in anyone with a rudimentary understanding of software engineering. >You hit the >nail on the head. This is *exactly* what I am trying to avoid. >Besides, if we didn't cherish information hiding, language protection >and ease of uses why don't we all go back to programming in assembly >language? Because of "ease of use and language protection". Ease of use and information hiding are orthogonal (and in my biased opinion, inversely proportional). >We need language features to help in providing protection and ease of >use. Not in Common Lisp. What other language provides DISASSEMBLE as a standard function? > >OK. Here you say that CLOS doesn't enforce any barrier. I agree. Also >you are right. Lisp isn't into firewalls. This is my complaint. Strict information hiding insures that only the original developers can debug and improve the code. That's not a desirable state of affairs. Tim Moore moore@cs.utah.edu {bellcore,hplabs}!utah-cs!moore "Ah, youth. Ah, statute of limitations." -John Waters
miller@cam.nist.gov (Bruce R. Miller) (10/19/90)
In article <271E0D40.451E@wilbur.coyote.trw.com>, Scott Simpson writes: > In article <2865200060@ARTEMIS.cam.nist.gov> miller@cam.nist.gov writes: > ... > I agree that the intent that I intended to use packages for and the > intent they were designed for are probably quite different. I don't > know what packages were designed for. I have been told that they are > to be used for separating large subsystems of code. As far as I know, that's exactly what they're for. > The point I have > been trying to make is that the information hiding granularity of > packages is way too coarse and no other mechanism in Lisp can give me > the granularity I want (i.e., class based) easily. I don't find > packages very useful and I wouldn't cry a tear if they were simply > removed from the language. Just because packages doesn't solve the particular problem of the moment doesn't mean they're not useful. I would miss them. > I don't know if I agree that CLOS deals > with information hiding. How do you mean? Exactly in this sense: > >One point is that, in Lisp, nothing is REALLY hidden -- some things are > >just a little less visible than others. Want a non-exported symbol from > >a package? Use two colons. Want to get at an unapproved slot from a > >clos instance? Just use slot-value. > > Yes. I hate this. The double colon is way too similar to the single > colon. I can concede that for efficiency reasons or some such it is > non unreasonable to have back doors that can break the abstraction > barrier. ... Well, I dont mind it a bit. Double colons are noticable to me. I surely notice when I type them in! Ditto slot-value. And although given what Lisp is, it would surely introduce inefficiencies to try to introduce firewalls, I seriously doubt that that has anything to do with the lack of them. Who wants them? (besides you I mean :> ) > >On the one hand, I think that packages are (very) useful at a coarse > >grain: for example at the application level. I've never written an > >application with more than 2 packages and even those were cases where > >the one might be used as a separate set of tools. [I do often write > > If you hardly ever use packages, why do you find them so useful? ? I ALWAYS use packages -- I just dont find myself needing to use more than 2 in any given app. I find them useful for just what I said: separating applications and gross layers within an application. > > ... You just dont call the internals by virtue of your strength of > >will and extraordinary discipline! > > Exactly. Strength of will and extraordinary discipline. You hit the > nail on the head. This is *exactly* what I am trying to avoid. Why? Besides, I like being virtuous!:> > Besides, if we didn't cherish information hiding, language protection > and ease of uses why don't we all go back to programming in assembly > language? They all are Turing machine equivalent anyway. Right? I > don't think that the difference between external and internal > functions should just be that one is documented and the other isn't. > We need language features to help in providing protection and ease of > use. FMHT (for my humble tastes), I think you go a bit too far here. > >As you've seen, using too many packages within the same program gets > >hairy. But, I dont think that CLOS really attempts to enforce any kind > >of barrier, and packages seems to be at the wrong level for what you > >want --- lisp itself isn't really into firewalls in any case. > > OK. Here you say that CLOS doesn't enforce any barrier. I agree. Also > you are right. Lisp isn't into firewalls. This is my complaint. Emphasis on ENFORCE. I guess if you REALLY want that much of a barrier, you've just got to use a multitude of packages and a multitude^2 of exports and not USE the packages within each other. You've got to say what is OK and not OK anyway -- what's wrong with export? > I think most Lisp users suffer in the same way. > Scott Simpson TRW scott@coyote.trw.com I'm certainly not `most' but I'm not suffering. bruce - - - - -- - - dumb posting software - - - - - -
mbr@flash.bellcore.com (Mark Rosenstein) (10/19/90)
In article <271E0D40.451E@wilbur.coyote.trw.com> scott@wiley.uucp (Scott Simpson) writes:
From: scott@wiley.uucp (Scott Simpson)
Newsgroups: comp.lang.lisp
Summary: I'm not convinced.
I agree that the intent that I intended to use packages for and the
intent they were designed for are probably quite different. I don't
know what packages were designed for. I have been told that they are
to be used for separating large subsystems of code. The point I have
been trying to make is that the information hiding granularity of
packages is way too coarse and no other mechanism in Lisp can give me
the granularity I want (i.e., class based) easily. I don't find
packages very useful and I wouldn't cry a tear if they were simply
removed from the language.
Well, I'd be major depressed. Ain't you never had name conflicts in C
when you try to load multiple big subsystems? It's sort of hard to
imagine running in a world with flavors, CLOS and PCL all present
without packages, much less CLIM, CLM, CLX, and half a dozen user
packages. They can mostly all coexist, call each other, without much
fuss. [If one object system is good, two must be better, and three
well, you know, the old code that wouldn't die]. I tend to run a given
lisp world for weeks. By the end, I have metering loaded and this and
that, and suprise, suprise they don't interfer with each other. I shudder
to think what would happen if all their symbols were in the user package.
I expect one uses packages to avoid conflict with other subsystems that
you have no control over as well as to segment your program. I don't
know that there is a good treatment, or even if one is possible on how
to segment functionality along package lines. For small systems one will
do and bigger stuff, I've used up to say a dozen, but that might have been
a few too many.
Yes. I hate this. The double colon is way too similar to the single
colon.
They don't seem that similar to me. Everytime I use a double colon,
the good code fairie comes into my office and smacks me on the side
of the head. Unless of course, I'm calling some system function
that isn't documented, but I need. Then I just pay for it in the
next release when it changes under me. But I know. I do it to myself
of my own volition. I honestly have never mistakenly used a double
colon.
I can concede that for efficiency reasons or some such it is
non unreasonable to have back doors that can break the abstraction
barrier. C++ does this with its ugly friend functions. However, I do
think that they should be discouraged and if used, should be quite
noticeable. It is said that they put gotos in Ada (especially for
automatically generated Ada programs) but that they discourage their
use and make them stick out like a sore thumb. Ada's labels look like
<<label>>.
This is lisp, not Pascal. This is why all good little lisp programmers
read Abelson and Sussman or Allen.
Exactly. Strength of will and extraordinary discipline. You hit the
nail on the head. This is *exactly* what I am trying to avoid.
Besides, if we didn't cherish information hiding, language protection
and ease of uses why don't we all go back to programming in assembly
language? They all are Turing machine equivalent anyway. Right? I
don't think that the difference between external and internal
functions should just be that one is documented and the other isn't.
We need language features to help in providing protection and ease of
use.
Scott Simpson TRW scott@coyote.trw.com
Wow. Strength of will and extraordinary discipline. Heck, here I thought
it was plain economics. Well designed code is cheaper over its lifetime
than poorly designed code. [Go ahead do a study on that. Step 1: get
a definition of well designed code. (you don't need step 2, step 1
will never return)].
Ok. I've actually lost the thread of this information hiding stuff. Keene's
book on CLOS has many excellent examples of the use of CLOS to insulate
interfaces from underlying implementation. It isn't a firewall. For
debugging, for quick hacks, for system code, lisp has always given you
at least ten ways to screw yourself to the wall (redefine cons with your
more efficient version, say).
This is not to say that I don't want my compiler or my environment to
warn me when I am doing something unusual (like when two packages
have the same nickname---waaaah), but I never want my lisp to think it
is smarter than me. It can ask "Do you really want to screw yourself?",
but it darn well better let me, when I answer "Yes, thanks." And certainly
runtime checking on array bounds, say, is a major win. I guess fundamentally
good design isn't easy. I want major support from my language and from my
environment, but sometimes a programmer's got to do what a programmer's
got to do.
Mark.aihaug@AUSTIN.LOCKHEED.COM (Daniel A Haug) (10/19/90)
Last year, we completed a project that involved several hundred thousand
lines of CLOS-intensive Lisp code. Just guessing, I'd say that we had
around 1000 classes, and perhaps 2000 generic functions. We used the
package system to isolate layered functionality. For example, the
CLOS-based window system was in one package, the CLOS-based user-interface
management system was in another package, the CLOS-based communications
system in yet another. You get the idea.
In each system, public class slots were exported. So were public interface
functions. Private slots and functions were not exported. No big deal.
Also, I didn't understand the earlier comment about how defclass generates
all of these other hidden functions (?). I do not know what functions
outside of the CLOS-specified functions are generated.
(defclass a ()
((b :initform nil
:accessor a-b)))
This only generates one new method: a-b. All of the other functions are
part of the CLOS spec (e.g. make-instance, initialize-instance, slot-value).
Now back to how one would use our system: If I were then writing an
application that would use the communications system, the window system,
and the UIMS, then I would create my own application package that uses
those packages:
(make-package :my-application
:use '(:common-lisp
:clos
:window-system
:uims
:communications))
Then my application package would inherit all of the public slots and
interfaces from these subsystems. Then suppose that the window-system
has class BASIC-WINDOW with public slot BACKGROUND-COLOR, and that
all of my application windows will have blue background colors (back
in the old pre-X days of hardcoded display preferences ;-), I would write:
(defclass my-app-window (basic-window)
((background-color :initform :blue)))
Thus, it is fairly easy to generate subclasses based on public (read: exported)
class information. I have not violated object boundaries. My application
window class doesn't interfere with other users of :window-system.
Now, in practice, since we had such a large system, this approach worked
well 99% of the time. Occasionally we had name conflicts arise within
the package. No big deal... find it and resolve it. Also, we would have
occasional package conflicts, two subsystem packages exporting similar
symbol names. Again, no big deal, just choose the desired symbol to inherit,
and shadow the others. Or, don't use :USE-PACKAGE at all:
(defclass my-app-window (window-system:basic-window)
((window-system:background-color :initform :blue)))
To me, there is flexibility to suit many approaches to this.
From article <271E0D40.451E@wilbur.coyote.trw.com>, by scott@wiley.uucp (Scott Simpson):
>
> [...] I don't find
> packages very useful and I wouldn't cry a tear if they were simply
> removed from the language. I don't know if I agree that CLOS deals
> with information hiding. How do you mean?
No! We would have DIED in our project without the package system. We
probably had 50 packages in the end. I can't begin to imagine life
without the package system! Having done a substantial amount of work
in C, which has no real namespace support (other than: everyone gets it,
or no-one outside of this file gets it), I am very appreciative of
languages that provide such a facility (e.g. Lisp, Ada, Modula-II,...).
In C, one is forced to use implicit namespace dividers by prepending
a pseudo-system name on each symbol (e.g. Xt, X, Xm, Xaw,...) which
is pretty ridiculous (IMHO).
My only serious gripe about the package system is the inability (within
pure CL) to lock a package. In our project, we had a few inexperienced
engineers who would develop the window-system side of their application
in the window-system package. As their application would be loaded
late into the lisp world, their additions to the window-system package
would suddenly be inherited by all other users of the window-system,
and everyone would indubitably lose. A nice safeguard would have been
to lock the window-system package after that system had been loaded.
Finally, to summarize my posting: We had success by using packages
to isolate subsystems, including class definitions and methods. As we
usually had only one or two people working on a system, name-conflicts
rarely arose. Public vs. Private class information was accomplished
via exporting symbols.
dan haug
--
Internet: haug@austin.lockheed.com
UUCP: ut-emx!lad-shrike!aihaug
Phone: (512) 448-5634lou@cs.rutgers.edu (lou) (10/19/90)
In article <1990Oct18.152453.7100@hellgate.utah.edu> moore%cdr.utah.edu@cs.utah.edu (Tim Moore) writes:
[...] I think it's great that data structures are exposed in Lisp,
if only to satisfy my own curiosity. [...] Strict information
hiding insures that only the original developers can debug and
improve the code.
There is another, more basic, reason that lisp folks have not in
general gone in for strict information hiding, i.e. with no escape
hatches like the double-colon. That is because lisp has been almost
exclusively used with interactive programming environments for quite a
while now. Code trying to access a non-exported variable of some
package may be due to a programmer "cheating" on the modularization of
the program, but it MAY simply be a programmer typing to a
read-eval-print loop trying to figure out what is happening with his
program. The only way a programmer has to interact with his running
system is to cause expressions to be eval'ed. If there is no way for
an *expression* to access something, then there is no way for the
*programmer* to access it during debugging. (I know, accessing stack
frames in Common Lisp appears to be an exception, but that was a
compromise for the sake of allowing a variety of underlying
implementation methods.)
--
Lou Steinberg
uucp: {pretty much any major site}!rutgers!aramis.rutgers.edu!lou
internet: lou@cs.rutgers.edujeff@aiai.ed.ac.uk (Jeff Dalton) (10/20/90)
In article <271CDC30.1E54@wilbur.coyote.trw.com> scott@wiley.UUCP (Scott Simpson) writes: >And if I have multiple classes within one package then I have >information hiding problems. Use a package for a "service" (ie, something that "clients" might use). Such facilities might involve several classes that logically go together. (That is, it makes sense to make them be part of the same service) A CLOS implementation would be one example. It contains a number of classes, but will involve only one (or a few) package(s). The information-hiding problems among different classes in the same poackage, when the package embodies a reasonable, logical grouping, are not serious enough to worry about -- at least not in Common Lisp. If you worry about them, then you will find that packages don't work very well for you. If you expect packages to give you the same capabilities you have in Eiffel or Ada, you are bound to be disappointed. -- Jeff
welch@sacral.cis.ohio-state.edu (Arun Welch) (10/20/90)
In article <271CDC30.1E54@wilbur.coyote.trw.com> scott@wiley.uucp writes: >I agree that the intent that I intended to use packages for and the >intent they were designed for are probably quite different. I don't >know what packages were designed for. I have been told that they are >to be used for separating large subsystems of code. Yup, that's it. To give an example of how packages are so usefull, look at Interlisp, which had no packages. The convention that everyone used was, to quote from a truly ancient mail message to 1100Users: >Date: 18 FEB 84 18:31 PST >From: ROACH.PA@PARC-MAXC.ARPA >Subject: RE: Flame topic: Naming Conventions and Packages >To: 1100USERS@SUMEX-AIM.ARPA >cc: ROACH.PA@PARC-MAXC.ARPA > > I would encourage the following convention. Atoms in a package >used for function names, property names, or as global variables should >have the form P.FOO or \P.FOO where P is a nontrivial (>= 3 chars) >prefix. . . . > Slash with "\" any function name, property name, >or global variable whose casual redefinition via DEFINEQ, PUTPROP, or >SETQ (you get the idea) would crash the user. Read module for package above, since the message predates the current use of package. Believe me, having packages is a whole lot better than the other option. Doing an apropos on something like OPEN under Medley yeilds FB.OPEN, TCP.OPEN, CHAT.OPEN, etc. It's interesting to see that some people are even using pseudo-packages in Emac's Elisp, by prefixing their code with package-name:variable (Franz's lisp-emacs interface comes to mind). Most lisps are really whole environments, and you really don't want to clobber yourself by redefining something as innocuous as VERIFY, for example, if it was used by the memory management system. IL programmers generally went to great pains so that casual users didn't redefine important symbols, and having a package boundary with import and export is much simpler. I'd suspect that most IL programmers have trashed the environment at least once by not being careful enough, and having to reload and/or recreate your work isn't fun. ...arun ---------------------------------------------------------------------------- Arun Welch Lisp Systems Programmer, Lab for AI Research, Ohio State University welch@cis.ohio-state.edu
scott@wilbur.coyote.trw.com (Scott Simpson) (10/20/90)
In article <1990Oct18.152453.7100@hellgate.utah.edu> moore%cdr.utah.edu@cs.utah.edu (Tim Moore) writes: >I don't see what the problem is. If you don't want to get burned by >using internal functions and data structures, don't use them. I think >it's great that data structures are exposed in Lisp, if only to >satisfy my own curiosity. One of the first things I do when I use a >Common Lisp implementation for the first time is type (symbol-plist '+) >to see what turns up. (Maybe that comes after (time (fact 1000)) :-) I don't think that back doors aren't useful, I am just concerned about them being the default. >That's not what he said at all. The basic idea behind packages is that >it's not too hard to avoid name collisions in your own code, or even >within an "application" (you can use grep, after all), but it is hard >to avoid collisions with other random programs that might be loaded in >the same Lisp image. For one program, one or two packages is about right. I don't think grep is a great way to discover name collisions! Surely we can come up with something better. Browsers perhaps? >Because of "ease of use and language protection". Ease of use and >information hiding are orthogonal (and in my biased opinion, inversely >proportional). Not necessarily. I see your point but I am not so black and white on the issue. In article <359@skye.ed.ac.uk> jeff@aiai.ed.ac.uk (Jeff Dalton) writes: >If you expect packages to give you the same capabilities you >have in Eiffel or Ada, you are bound to be disappointed. Yes. I have come to this conclusion. Since I don't want to beat a dead horse, I will drop this thread after this message. -- Scott Simpson TRW scott@coyote.trw.com
kers@hplb.hpl.hp.com (Chris Dollin) (10/22/90)
I've been following this with some interest. At least one person has remarked that "they only need one or two packages in an application". Then Daniel A Haug says: Last year, we completed a project that involved several hundred thousand lines of CLOS-intensive Lisp code. Just guessing, I'd say that we had around 1000 classes, and perhaps 2000 generic functions. We used the package system to isolate layered functionality. For example, the ... No! We would have DIED in our project without the package system. We probably had 50 packages in the end. I can't begin to imagine life Several 100K lines of source; 50 packages. When we built Axis (Axis [*1] is an equational-specification language & system, similar in intent to the OBJ family) we had about 27K lines of source (yes, just twenty-seven thousand) and 110 packages (I just counted). Are we that unusual? To us it seemed natural to build our system as a collection of implementations of abstract data-types, and make each module a package; lots of datatypes => lots of packages. We'd wouldn't have done without the package system, although (a) not having structures package names was a pain, (b) not having renamed-imports would be better, (c) packageing seems to be done in the wrong place - as part of the lexicalandsyntactic analysis, not in the compiler [*2]. [*1] See my paper in the Europal 90 proceedings for a brief description of Axis and its implementation. [*2] Or interpreter. For these purposes the difference is irrelevant. -- Regards, Kers. | "You're better off not dreaming of the things to come; Caravan: | Dreams are always ending far too soon."
jeff@aiai.ed.ac.uk (Jeff Dalton) (10/23/90)
In article <271E0D40.451E@wilbur.coyote.trw.com> scott@wiley.UUCP (Scott Simpson) writes: > I don't >know what packages were designed for. I have been told that they are >to be used for separating large subsystems of code. The point I have >been trying to make is that the information hiding granularity of >packages is way too coarse and no other mechanism in Lisp can give me >the granularity I want (i.e., class based) easily. That's right: Common Lisp doesn't have such a mechanism. So what are we supposed to do now? (BTW, you could write a mechanism, but anyone who knew how it worked could probably get around it with some effort. Would that be good enough or do you want more?) >I don't find packages very useful and I wouldn't cry a tear if they >were simply removed from the language. So don't use them. I don't quite understand what you're trying to accomplish. Are you hoping Common Lisp will change, that the Common Lisp designers will be embarrassed at their incompetence, that Common Lisp will tossed in the trash, or what? >>One point is that, in Lisp, nothing is REALLY hidden >Yes. I hate this. Different people hate different things. That's why the world contains more than one programming language, and, indeed, more than one Lisp. >>On the one hand, I think that packages are (very) useful at a coarse >>grain: for example at the application level. I've never written an >>application with more than 2 packages and even those were cases where >>the one might be used as a separate set of tools. [I do often write > >If you hardly ever use packages, why do you find them so useful? He didn't say he hardly ever used them, but rather that he never wrote an application that involved more than two. >>You just dont call the internals by virtue of your strength of >>will and extraordinary discipline! > >Exactly. Strength of will and extraordinary discipline. You hit the >nail on the head. This is *exactly* what I am trying to avoid. >Besides, if we didn't cherish information hiding, language protection >and ease of uses why don't we all go back to programming in assembly >language? They all are Turing machine equivalent anyway. Right? Turing-equivalence is almost useless as a way to compare programming languages. Information-hiding is just one of the things higher-level languages can do, and not necesarily the most important. If it's the most important to you, or for the work you're doing, then don't use Common Lisp. It's as simple as that, isn't it? >Lisp isn't into firewalls. This is my complaint. I would have thought a shorter message would have sufficed for that. -- JD
jeff@aiai.ed.ac.uk (Jeff Dalton) (10/23/90)
In article <WELCH.90Oct19155545@sacral.cis.ohio-state.edu> welch@sacral.cis.ohio-state.edu (Arun Welch) writes: >Yup, that's it. To give an example of how packages are so usefull, >look at Interlisp, which had no packages. The first function I ever defined in InterLisp-D redefined something in the editor. The function name I used was (if I recall correctly) DOT. However, it's worth noting that the designers of other Lisp dialects (eg, Scheme) do not seem to be planning to adopt Common Lisp's packages. Much of the Prolog community has also rejected that approach. (The so-called atom-based proposals for modules get so much criticism from some quarters that I don't think one could ever be adopted as standard.) And it's hard to see many other langauges doing it, because other languages usually don't have symbols as a datatype. Packages do have problems. Just for example: suppose you load in some code but forget to put in a required USE-PACKAGE. Well, all the symbols that ought to be USED-PACKAGE:SYM will be LOCAL-PACKAGE::SYM, and that's very hard to fix once it's happened. However, packages are a fairly good "pragmatic" solution in that they work fairly well and are available now, while the better Scheme solutions are still being developed.
scott@wiley.uucp (Scott Simpson) (10/23/90)
(use-package 'flame) In article <3596@skye.ed.ac.uk> jeff@aiai.UUCP (Jeff Dalton) writes: >So don't use them. I don't quite understand what you're trying to >accomplish. Are you hoping Common Lisp will change, that the Common >Lisp designers will be embarrassed at their incompetence, that Common >Lisp will tossed in the trash, or what? I am trying to get information hiding semantics like Eiffel and Ada in Lisp without too much effort. I have been told by many people I can't. I believe them. I think you are being a tad touchy here. I guess every language news group has its language zealots... >>Exactly. Strength of will and extraordinary discipline. You hit the >>nail on the head. This is *exactly* what I am trying to avoid. >>Besides, if we didn't cherish information hiding, language protection >>and ease of uses why don't we all go back to programming in assembly >>language? They all are Turing machine equivalent anyway. Right? > >Turing-equivalence is almost useless as a way to compare programming >languages. This is exactly what I said! I asked a rhetorical question. Reread the paragraph starting with "Besides". You also edited my paragraph and I fear it could be taken out of context. The last sentence of the above paragraph stated "We need language features to help in providing protection and ease of use." >I would have thought a shorter message would have sufficed for that. My apologies for not being so succinct. P.S. I *really* promise not to post after this! Scott Simpson TRW scott@coyote.trw.com
jeff@aiai.ed.ac.uk (Jeff Dalton) (10/24/90)
In article <272381E7.44AA@wilbur.coyote.trw.com> scott@wiley.UUCP (Scott Simpson) writes: >In article <3596@skye.ed.ac.uk> jeff@aiai.UUCP (Jeff Dalton) writes: >>So don't use them. I don't quite understand what you're trying to >>accomplish. Are you hoping Common Lisp will change, that the Common >>Lisp designers will be embarrassed at their incompetence, that Common >>Lisp will tossed in the trash, or what? > >I am trying to get information hiding semantics like Eiffel and Ada in >Lisp without too much effort. I have been told by many people I >can't. I believe them. I think you are being a tad touchy here. I >guess every language news group has its language zealots... I just think that, after a while, a discussion ought to have some kind of goal. At first I thought you were trying to find the best way to use packages and classes together. Packages are far from perfect, but it makes sense to try to make the best of them. Later, it seemed that what you wanted was a way to get the same facilities as Ada or Eiffel. That too seems reasonable, as far as it goes. It turned out that there wasn't a good way to get such facilities (at least no one has suggested one), but it was worth trying. But then it looked like you'd decided to attack the way Common Lisp did things, pointing out that it was very unsatisfactory from the Ada/Eiffel point of view. Well, fair enough; but at that point it became less clear what practical result you aimed to accomplish. I happen to think that different languages are good for different things and that there isn't much point to "A is better than B" language wars. I probably should have written a milder article, but that's not the way it came out. There's just no way Common Lisp can satisfy everyone in any case. >>>Exactly. Strength of will and extraordinary discipline. You hit the >>>nail on the head. This is *exactly* what I am trying to avoid. >>>Besides, if we didn't cherish information hiding, language protection >>>and ease of uses why don't we all go back to programming in assembly >>>language? They all are Turing machine equivalent anyway. Right? >> >>Turing-equivalence is almost useless as a way to compare programming >>languages. > >This is exactly what I said! I asked a rhetorical question. Reread the >paragraph starting with "Besides". As far as I can tell, what you had in mind was either: 1. If we don't cherish A, B, and C, we may as well use assembler. 2. People who don't cherish A, B, and C don't understand what higher-level languages are for. > You also edited my paragraph and I >fear it could be taken out of context. The last sentence of the above >paragraph stated "We need language features to help in providing >protection and ease of use." I edited the quotes to keep down message size. I didn't ignore the rest and thought what I kept was representative. However, if you think I cut out too much, I accept that and apologize. Common Lisp has at least two ways to make information relatively inaccessible. One is packages. Packages work fairly well for some things, but they don't provide the strong inacessibility that some people want. The other is lexical scoping. Lexical scoping does provide fairly strong inaccessibility, because there are no facilities in the language for looking inside closures. So GENERIC-LABELS would let you hide generic functions, in a sense. >P.S. I *really* promise not to post after this! I hope you do post. Most of what you said about packages was well worth reading. -- Jeff
jeff@aiai.ed.ac.uk (Jeff Dalton) (10/24/90)
In article <KERS.90Oct22090640@cdollin.hpl.hp.com> kers@hplb.hpl.hp.com (Chris Dollin) writes: >(c) packageing seems to be done in the wrong place - as part of the lexical >and syntactic analysis, not in the compiler [*2]. >[*2] Or interpreter. For these purposes the difference is irrelevant. Continuing in my current role as Defender of Common Lisp... Packages are handled by the reader. Thus they apply to data as well as code. This approach has some disadvantages (see one of my earlier articles for an example), but it also has advantages. There are several ways in which data structures can end up being interpreted as code: 1. A symbol that has a global function definition can be used as a function. 2. A list, symbol, or other object can be given to EVAL. 3. A list, symbol, or other object can result from macro expansion. Let's look at the first example. If the package information were not part of the symbol, and the symbol were given as a function to MAPCAR, MAPCAR would have no way to determine which package held the function definition. [Why doesn't MAPCAR know? The symbol doesn't say, and the compiler can't find all calls to MAPCAR and stick the information in at the point of call. The reason the compiler can't find all calls is that some might be done via FUNCALL or APPLY or some function that calls FUNCALL or APPLY. Not to mention EVAL. Moreover, for user-defined functions, the compiler might not be able to determine which arguments were functions (and so needed package information inserted at the call).] Similar problems occur if a list containing function names is given to EVAL or returned as the result of macro expansion. There are several solutions that might be applied to these problems: S-1. Eliminate the offending parts of the language. S-2. Adopt a convention such as: always use the current package. S-3. Have the user explicitly specify the package. S-4. Invent a zowie new mechanism that makes (almost) everything work as we (ought to) expect. Some people favor (S-1) for (1) and (2). That is, eliminate the ability to use symbols as functions and eliminate EVAL. Some people would apply it to (3) as well: no macros. That seems a rather large price to pay. As for S-2, it's difficult to come up with a convention that works in enough cases. On the other hand, it might be argued that we should eliminate (1) and that "use the current package" is good enough for (2) and (3). I think it isn't good enough for (3) [or for (1)]. That is, I should be able to write a macro like this one: (defmacro mac (x) `(some-package:some-function ,x)) or one like this: (defmacro mac (x) `(some-function ,x)) where I expect package information to stick to SOME-FUNCTION. [N.B. I haven't said what "current package" means. I don't think it works well enough to use either the current value of *package* or the current "compilation" package: ie, the package in which the code being compiled is defined. Not does it work to always use the package in which the macro was defined.] S-3 is often used for EVAL: give EVAL an extra "environment" argument. It would be a pain to use it for (1), giving an extra argument to MAPCAR and all the rest. Moreover, it's hard to see how it would work for (3). So that leaves S-4. It is clearly the best solution, but rather difficult to devise. The Scheme world will eventually do it for modules and macros in Scheme. Scheme won't support symbols as functions, and it won't have EVAL. Scheme, then, will offer a reasonable set of solutions. But (a) some regard it as a high price to pay, and (b) the Common Lisp solution, imperfect as it is, is available now and, moreover, was available back when people were writing whatever applications we're using now. -- Jeff
kers@hplb.hpl.hp.com (Chris Dollin) (10/24/90)
In response to my: >(c) packageing seems to be done in the wrong place - as part of the lexical >and syntactic analysis, not in the compiler [*2]. >[*2] Or interpreter. For these purposes the difference is irrelevant. Jeff posts quite a lot, from which I extract below. Packages are handled by the reader. Thus they apply to data as well as code. This approach has some disadvantages (see one of my earlier articles for an example), but it also has advantages. I have found it inconvenient that the pacakging system applies to data. I have resorted to using keywords rather than quoted "plain" symbols in several places just to ensure that packaging doesn't confuse me! [I've not seen Jeff's earlier article. Jeff, could you mail it to me?] There are several ways in which data structures can end up being interpreted as code: I must declare my colours. In general I don't like to treat data as code. This bias will show up below. 1. A symbol that has a global function definition can be used as a function. I would claim that the symbol "should" have been converted to its functional value at the point it was passed. That conversion must, of course, respect packaging. 2. A list, symbol, or other object can be given to EVAL. This would require that sufficient information be given to EVAL for it to resolve packaging in the same way that the compiler (or interpreter) would. That seems reasonable to me. 3. A list, symbol, or other object can result from macro expansion. I don't think that matters; if packaging is a property of the compiler, then macros would have to respect it; they might have to embed additional package *syntax* in their result, but I don't think this is a big deal. Let's look at the first example. If the package information were not part of the symbol, and the symbol were given as a function to MAPCAR, MAPCAR would have no way to determine which package held the function definition. I've hand-waved over that one above. I think if one were to take compiler packaging seriously, then the symbol would have been converted at an earlier appropriate moment. A more serious problem is that the symbol gives you a layer of indirection; between passing the symbol in, and applying it with MAPCAR (strictly, I suppose, FUNCALL or APPLY), the global binding might change. If the "old" value has been captured, but the new one intended, then I'm stuffed. A solution is to allow *variables* as function-objects, and to pass the variable corresponding to the symbol as argument; the package information would be resolved at this point. [In this model, I distinguish between a *symbol*, which is just an interesting data-object with no value, package, or function cells, and a *variable*, which has one value [*1] and possibly name and package cells]. Similar problems occur if a list containing function names is given to EVAL or returned as the result of macro expansion. Not if EVAL has the information that allows packages to be resolved passed in as an additional argument. There are several solutions that might be applied to these problems: S-1. Eliminate the offending parts of the language. S-2. Adopt a convention such as: always use the current package. S-3. Have the user explicitly specify the package. S-4. Invent a zowie new mechanism that makes (almost) everything work as we (ought to) expect. I suspect Jeff would say I've picked S-4. Chopping context ... ... That is, I should be able to write a macro like this one: (defmacro mac (x) `(some-package:some-function ,x)) or one like this: (defmacro mac (x) `(some-function ,x)) Well, I'd write them the same way! But the reader item "package:id" would be translated as (eg) "(inpackage package id)", where "inpackage" is a special form which resolves to the variable called "id" in the package "package". S-3 is often used for EVAL: give EVAL an extra "environment" argument. It would be a pain to use it for (1), giving an extra argument to MAPCAR and all the rest. Moreover, it's hard to see how it would work for (3). I've already indicated my solution for MAPCAR; I agree with Jeff that insisting that all things that take functional arguments that might be symbols (or variables) would be a right pain. So that leaves S-4. It is clearly the best solution, but rather difficult to devise. The Scheme world will eventually do it for modules and macros in Scheme. Scheme won't support symbols as functions, and it won't have EVAL. I think I've sketched an interim solution which would work as effectively as pacakges do now, and gives what I regard as several advantages: eg, data is immune from the package system (so [modulo UNINTERN and its ilk] there's only one symbol with a given printname; one can write macros which do package manipulation, such as "(in-a-package Package Form*)" which compiles the Form's in the context of that package. I'm not seriously suggesting that it's a candidate to go into CL; it's far too late for that. But I think that it demonstrates that there is a rational alternative to read-time packaging. Scheme, then, will offer a reasonable set of solutions. But (a) some regard it as a high price to pay, and (b) the Common Lisp solution, imperfect as it is, is available now and, moreover, was available back when people were writing whatever applications we're using now. Well, I agree with that. I just wish the rational reconstruction was what we'd been given in the first place! -- Jeff [Hey, Jeff, I didn't mention [omitted] once!] -- Regards, Kers. | "You're better off not dreaming of the things to come; Caravan: | Dreams are always ending far too soon."
jeff@aiai.ed.ac.uk (Jeff Dalton) (10/26/90)
In article <KERS.90Oct24101043@cdollin.hpl.hp.com> kers@hplb.hpl.hp.com (Chris Dollin) writes: > >(c) packageing seems to be done in the wrong place - as part of the > >lexical and syntactic analysis, not in the compiler [*2]. > >[*2] Or interpreter. For these purposes the difference is irrelevant. >Jeff posts quite a lot, from which I extract below. > > Packages are handled by the reader. Thus they apply to data > as well as code. This approach has some disadvantages (see > one of my earlier articles for an example), but it also has > advantages. >I have found it inconvenient that the packaging system applies to >data. I have resorted to using keywords rather than quoted "plain" >symbols in several places just to ensure that packaging doesn't >confuse me! You have to arrange for the right data symbols to be imported, exported, etc. There are times when this can be a pain, but there are other times when it would be a pain to have all symbols in one package. > There are several ways in which data structures can end up > being interpreted as code: >I must declare my colours. In general I don't like to treat data as code. >This bias will show up below. Lisp has traditionally been a language in which you can do such things. Scheme has not. I think it's a good thing that both approaches are represented. > 1. A symbol that has a global function definition can be > used as a function. > >I would claim that the symbol "should" have been converted to its functional >value at the point it was passed. That conversion must, of course, respect >packaging. There are, no doubt, ways to do this without losing track of what symbol it was. (Just taking the functional value would be to lose track.) You suggest, below, that the symbol might be converted to a "variable". Maybe that would be a good solution, but it's hard to be sure without implementing it and seeing how it turns out. > 2. A list, symbol, or other object can be given to EVAL. > >This would require that sufficient information be given to EVAL for it to >resolve packaging in the same way that the compiler (or interpreter) would. >That seems reasonable to me. I agree. Indeed, that's how the Scheme implementations that have EVAL tend to handle it. > 3. A list, symbol, or other object can result from macro > expansion. > >I don't think that matters; if packaging is a property of the compiler, then >macros would have to respect it; they might have to embed additional package >*syntax* in their result, but I don't think this is a big deal. It is difficult to make this work in a reasonable way. Suppose, for example, you write a macro that defines a macro. It may need to put in some package information but not be able to tell what package it's being expanded in. Here's an example. DEFINE-FEXPR is used to define a "function" that operates on its unevaluated arguments (as a single list). What it actually defines is a macro that expands to call a function with all the arguments in a QUOTE. (defmacro define-fexpr (name (var) &body forms) (let ((fn-name (gentemp (concatenate 'string (string name) "-FUNCTION")))) `(progn (defun ,fn-name (,var) ,@forms) (defmacro ,name (&rest ,var) `(,',fn-name ',,var))))) And now the question is: how does DEFINE-FEXPR know what module the function is in so that the macro it defines can expand to a call to *that* function. (Despite our GENTEMP, there may later be another function with that name.) You also have to consider how macro calls are written. The caller shouldn't have to add more (explicit) package information than would be needed when calling a function or a built-in special form. There are ways to deal with these problems, but it requires work to figure them out, and some of the mechanisms required look like they will be complicated and hard to understand. > Let's look at the first example. If the package information > were not part of the symbol, and the symbol were given as a > function to MAPCAR, MAPCAR would have no way to determine > which package held the function definition. > >I've hand-waved over that one above. I think if one were to take compiler >packaging seriously, then the symbol would have been converted at an earlier >appropriate moment. I agree. But that would be doing something else rather than making this case work. So I think we'd have to try implementing the something else ans see how well it worked. >A more serious problem is that the symbol gives you a layer of indirection; >between passing the symbol in, and applying it with MAPCAR (strictly, I >suppose, FUNCALL or APPLY), the global binding might change. If the "old" >value has been captured, but the new one intended, then I'm stuffed. > >A solution is to allow *variables* as function-objects, and to pass the >variable corresponding to the symbol as argument; the package >information would be resolved at this point. Note that it's much easier to do this if the user has to write something like (VARIABLE <name>) than if it's supposed to happen automatically. > There are several solutions that might be applied to these problems: > > S-1. Eliminate the offending parts of the language. > S-2. Adopt a convention such as: always use the current package. > S-3. Have the user explicitly specify the package. > S-4. Invent a zowie new mechanism that makes (almost) everything > work as we (ought to) expect. > >I suspect Jeff would say I've picked S-4. Chopping context ... > > ... That is, I should be able to write a macro like this one: > (defmacro mac (x) `(some-package:some-function ,x)) > or one like this: > (defmacro mac (x) `(some-function ,x)) > >Well, I'd write them the same way! But the reader item "package:id" would be >translated as (eg) "(inpackage package id)", where "inpackage" is a special >form which resolves to the variable called "id" in the package "package". That handles the first definition of MAC, but not the second. A number of macros that are easy to express in Common Lisp will have to be cluttered with package information (unless there was a clever implementation that figured it all out automatically). And the macros's author may not know all of the package information when writing the macro. (See above.) You can start adding mechanisms to handle all this, but it's difficult to devise good ones. > So that leaves S-4. It is clearly the best solution, but > rather difficult to devise. The Scheme world will eventually > do it for modules and macros in Scheme. Scheme won't support > symbols as functions, and it won't have EVAL. > >I think I've sketched an interim solution which would work as effectively as >pacakges do now, and gives what I regard as several advantages: eg, data is >immune from the package system (so [modulo UNINTERN and its ilk] there's only >one symbol with a given printname; one can write macros which do package >manipulation, such as "(in-a-package Package Form*)" which compiles the Form's >in the context of that package. I'm not seriously suggesting that it's a >candidate to go into CL; it's far too late for that. But I think that it >demonstrates that there is a rational alternative to read-time packaging. Well, I don't like that solution. I've tried to write macros that way recently, in an "Uncommon Lisp", and it was a pain. I don't think it works as effectively as packages, because packages often get it right for you, and the cases that aren't addressed by packages (certain name capture problems) have been around for so long that people have gotten used to dealing with them. Moreover, I think uninterned symbols take us further from "only one symbol with a given name" than "only one symbol with a given name in a given package" does. (Of course, it depends on which cases one has in mind.) -- Jeff PS -- You asked me to mail you a copy of an earlier message. I don't know which one you want. (I mention this here, because mail doesn't seem to be working very well these days.)