bentz@bbn.com (Bryan A. Bentz) (12/28/90)
I have a problem using the closure mechanism in Common Lisp. I suspect that this may have been addressed before, but if so it was before I was reading this list (in which case then please just respond to me directly). Basically, I want to create a function within a specific environment, which I can pass around and FUNCALL later on (see, for example, pages 87,88 of CLtL). For example, (let ((a 'inner-a) (b 'inner-b)) #'(lambda () (print a) (print b) t)) should return a closure which I can later FUNCALL to see the result INNER-A INNER-B T This will work, with one annoying caveat: if, at the time the function is defined, either A or B is a special variable, then the LAMBDA will reference that value, not the value set up by the LET. That is, if I do: (defvar a 'outer-a) (let ((a 'inner-a) (b 'inner-b)) #'(lambda () (print a) (print b) t)) I will get a closure back which, should I FUNCALL, will print out OUTER-A INNER-B T which is not what I intended. The special binding of A is being used instead of the lexical binding I wanted. After some searching, I found that on the bottom of page 39/top of page 40 of CLtL it is said that closures are only closures over non-special variables. Thus this behaviour is to be expected. This is where I have difficulties. 1. First of all, when writing the code (often automatically), how can I tell if a variable has been declared as special? As far as I can see, there is no way to find this out. If I can't determine it, then there is no way to guarantee that a given closure will do what I expect it to do. This seems somewhat bizarre. It might be argued that one solution would be to insist that all special variables should have names like *name*. Unfortunately there is no way to inforce this; some programmer might want to use a variable to hold some structure fragment during debugging - and end up causing problems when code is loaded later on. The problems are horrific to try to find, because the code is in some sense correct; due to circumstance or file loading order it may just end up using the wrong binding. Another aspect of this is that I might really want to shadow special variables in the closure; let's say I wanted to create a closure to redraw a window at half its original size; I might end up with: (defvar *window-width* 100 "Width of the window in pixels") (defvar *window-height* 80 "Height of the window in pixels") (defvar *half-size-redraw* (let ((*window-width* (/ *window-width* 2.0)) (*window-height* (/ *window-height* 2.0))) (declare (special *window-width* *window-height*)) #'(lambda () (redraw-window-as-usual-function)))) "Closure to redraw the window at half of its size") Here I would want to temporarily rebind *window-width* and *window-height* to half their size (at closure definition time) and then call the usual window redraw functions. This seems like it would be a nice way to preserve some important initial state information within procedures. It won't work in Common Lisp. In the above example, which won't work anyway, the (declare (special *window-width* *window-height*)) is intended to give the flavor of what I would like; that is, if one were to evaluate the (redraw-window-as-usual...) form instead of returning a closure, then the bindings set up in the LET would be in effect. This is what I would have expected from the closure as well. 2. A solution might be to traverse the lambda expression, find all variables used, and replace them with GENSYMed equivalents to guarantee that none are special. This is painful and in some cases just doesn't work; for instance, in this not-too-clever example, I would like the closure to reference the lexical binding of C set up in HACK; it won't. (defun hack (arg) (let ((c 3)) (declare (special c)) (setq *func* (make-up-closure arg)))) (defun make-up-closure (thing) #'(lambda () (print c) (print thing))) There is no easy way for me to rename C in MAKE-UP-CLOSURE and find all lexical bindings of C in functions which might call it and do the same renaming there. This example is probably not realistic, and tosses in some other issues (such as making sure the right binding for THING is used), but of course often one is never sure in exactly what way a particular mechanism will be used in the future. I would like a way to create closures which is guaranteed to work, and which will resolve references at FUNCALL time just as they would be resolved at function definition time. 3. Conceptually and aesthetically, it seems that closures should be closures over the entire environment. (I'm not looking at this from the point of view of an implementer.) Is there a reason why the decision was made to handle only lexical bindings? 4. I'm sure there are ways to hack one's way around this; I could probably figure out how to tell if a variable is special in a given implementation, for instance. Yet this leads to non-portable code, and I hate to even start down that road. I realize that some of these issues are really with the language specification, and thus there may not be an answer other than "That's the way it is." Yet it seems that I must be missing something, or else there is no way to use the closure mechanism reliably. Any comments would be welcome. Bryan A. Bentz BENTZ@BBN.COM
barmar@think.com (Barry Margolin) (12/28/90)
In article <61763@bbn.BBN.COM> bentz@bbn.com (Bryan A. Bentz) writes: >After some searching, I found that on the bottom of page 39/top of page 40 >of CLtL it is said that closures are only closures over non-special >variables. Thus this behaviour is to be expected. This is where I have >difficulties. > >1. First of all, when writing the code (often automatically), how can I >tell if a variable has been declared as special? As far as I can see, there >is no way to find this out. If I can't determine it, then there is no >way to guarantee that a given closure will do what I expect it to do. This >seems somewhat bizarre. > >It might be argued that one solution would be to insist that all special >variables should have names like *name*. Unfortunately there is no >way to inforce this This is the standard solution that is used. In general, Common Lisp avoids mechanisms for enforcing rules or preventing programmers from screwing themselves; the language gives the programmer lots of rope, and you are expected not to hang yourself. In this particular case, the assumption is that you have some measure of control over the variable names used in your application. If you are working with another programmer who doesn't follow the same conventions as you, you can use packages to keep from stepping on each other's toes. Put your part in one package, and his part in another package, and then his special variables won't conflict with your lexical variables. ; some programmer might want to use a variable to hold >some structure fragment during debugging - and end up causing problems when >code is loaded later on. The problems are horrific to try to find, because >the code is in some sense correct; due to circumstance or file loading >order it may just end up using the wrong binding. I don't know about other people, but I rarely use DEFVAR to declare my debugging variables special. I simply type (setq foo <expression>). This sets the global value of FOO, but it doesn't declare it special. >Another aspect of this is that I might really want to shadow special >variables in the closure; let's say I wanted to create a closure to redraw >a window at half its original size; I might end up with: > > (defvar *window-width* 100 "Width of the window in pixels") > (defvar *window-height* 80 "Height of the window in pixels") > > (defvar *half-size-redraw* > (let ((*window-width* (/ *window-width* 2.0)) > (*window-height* (/ *window-height* 2.0))) > (declare (special *window-width* *window-height*)) > #'(lambda () > (redraw-window-as-usual-function)))) > "Closure to redraw the window at half of its size") This can be done by rebinding the variables in the closed function, e.g.: (defvar *half-size-redraw* (let ((half-window-height (/ *window-height* 2)) (half-window-width (/ *window-width* 2))) #'(lambda () (let ((*window-width* half-window-width) (*window-height* half-window-height)) (redraw-window-as-usual-function))))) >2. A solution might be to traverse the lambda expression, find all >variables used, and replace them with GENSYMed equivalents to guarantee >that none are special. This is painful and in some cases just doesn't >work; for instance, in this not-too-clever example, I would like the >closure to reference the lexical binding of C set up in HACK; it won't. > > (defun hack (arg) > (let ((c 3)) > (declare (special c)) > (setq *func* (make-up-closure arg)))) > > (defun make-up-closure (thing) > #'(lambda () (print c) (print thing))) > >There is no easy way for me to rename C in MAKE-UP-CLOSURE and find >all lexical bindings of C in functions which might call it and do the >same renaming there. You've rediscovered why lexical variables are preferred over special variables. The renaming you suggest is precisely what a Lisp compiler does internally. And the inability to find all dynamic references to a variable is why it is only done for lexical variables. This example is probably not realistic, and >tosses in some other issues (such as making sure the right binding for >THING is used), but of course often one is never sure in exactly what >way a particular mechanism will be used in the future. I would like a >way to create closures which is guaranteed to work, and which will >resolve references at FUNCALL time just as they would be resolved at >function definition time. Zetalisp had dynamic closures; the function to create them took a list of symbols, and saved the dynamic bindings of those symbols in the closure. This mechanism was rarely used, so it was not included in Common Lisp (most of the uses of it were to make up for the fact that Zetalisp didn't have lexical closures). Dynamic closures are also hard to implement efficiently: in Lisps with shallow binding, calling a dynamic closure requires O(N) work, where N is the number of dynamic variables closed over, to rebind the closed variables; in deep-binding Lisps, calling a dynamic closure is fast, but references to dynamic variables (implemented by searching an alist) can be arbitrarily expensive. In fact, by using a variant of the example I gave above, you can implement Zetalisp's dynamic closure mechanism: (defun make-dynamic-closure (variables function) (let ((values (mapcar #'symbol-value variables))) #'(lambda () (progv variables values (funcall function))))) >3. Conceptually and aesthetically, it seems that closures should be >closures over the entire environment. (I'm not looking at this from >the point of view of an implementer.) Is there a reason why the decision >was made to handle only lexical bindings? This is a good point. Scheme closures include much more of the environment than Common Lisp closures (but Scheme also does without special variables). Common Lisp has more limited environments, mostly for pragmatic reasons. >I realize that some of these issues are really with the language >specification, and thus there may not be an answer other than "That's the >way it is." Yet it seems that I must be missing something, or else there >is no way to use the closure mechanism reliably. Any comments would be >welcome. In general, special variables are not supposed to be used heavily. It is well-documented in the S/W engineering literature that global variables are a source of problems, and Lisp special variables actually exacerbate the problems in some ways. Common Lisp doesn't supply features that would encourage their abuse. If Lisp's limitations on special variables are causing you problems, you're probably overusing them. Special variables were originally used to reduce the number of parameters that must be passed, especially when the eventual user of some parameters is several nested function calls down. Early Lisps didn't have and efficient mechanisms for passing optional and keyword arguments, so keeping argument lists small was important. Now we have :KEYWORD and &REST parameters, which make it much easier to use parameters for these things. A good example of this transition is the PRIN1 and WRITE functions; the latter was introduced into Common Lisp as a replacement for the former, converting all the special variable parameters into keyword arguments. -- Barry Margolin, Thinking Machines Corp. barmar@think.com {uunet,harvard}!think!barmar
davis@barbes.ilog.fr (Harley Davis) (12/28/90)
In article <61763@bbn.BBN.COM> bentz@bbn.com (Bryan A. Bentz) writes:
[Special variables are intermingled illegibly with lexical variables]
Some other Lisp dialects solve this problem simply by using separate
binding and referencing facilities for special (aka fluid or dynamic)
variables and lexical variables. For example, Le-Lisp version 16 and
EuLisp use the special binding form DYNAMIC-LET and the referencing
form DYNAMIC, and some implementations of Scheme use FLUID-LET.
In Le-Lisp version 16, for example, you could write:
(defvar var1 'outer-var1)
(defglobal my-closure
(let ((var1 'inner-var1))
#'(lambda () (print var1) (print (dynamic var1)))))
(funcall (global my-closure))
(dynamic-let ((var1 'new-var1))
(funcall (global my-closure)))
And the result would be:
inner-var1
outer-var1
inner-var1
new-var1
Declarations as they are done in CommonLisp are a parser's and
maintainer's nightmare, but solutions exist.
-- Harley
--
------------------------------------------------------------------------------
Harley Davis internet: davis@ilog.fr
ILOG S.A. uucp: ..!mcvax!inria!ilog!davis
2 Avenue Gallie'ni, BP 85 tel: (33 1) 46 63 66 66
94253 Gentilly Cedex
France