moore%defmacro.utah.edu@cs.utah.edu (Tim Moore) (04/03/91)
In article <14172@medusa.cs.purdue.edu> yeh@cs.purdue.EDU (Wei Jen Yeh) writes: >(defun monitor_1fun (fname) > (if (fboundp fname) > (if (get fname 'monitored) > (format T "Function ~A is already being monitored." fname) > (let ((old_fun (symbol-function fname))) > (setf (get fname 'monitored) old_fun) > (setf (symbol-function fname) > #'(lambda (&rest args) > (start_bench fname) > (multiple-value-prog1 (apply old_fun args) > (end_bench fname)))) >; (compile fname) > (setq *Monitor_List* (cons fname *Monitor_List*)) > *Monitor_List*)) > (format T "function ~A is not defined." fname))) > >However, the commented compile call failed. Is there any reason (both >in theory/implementation) why a lambda-closure cannot be compiled? Why >doesn't the standard (ep, ip) trick work? If monitor_1fun is compiled, then the closure is also already compiled. The result of calling COMPILE on a compiled function is unspecified. (Indeed, the call to COMPILE is unecessary.) However, if monitor_1fun is interpreted, then you are trying to compile an interpreted closure. Although CLtL1 didn't say much on this, X3J13 voted to not allow this in portable programs. From CLtL2 (pg 677): "It is an error if the function to be compiled was defined interpretively in a non-null lexical environment. (An implementation is free to extend the behavior of COMPILE to compile such functions properly, but portable programs may not depend on this capability.)" It should be obvious why compiling a closed "lambda expression" (as if there was any such thing) doesn't work; same reason that they can't be EVAL'ed. There are several problems that come up when compiling interpreted closures. The biggest one I can think of is dealing with the object that represents the closed-over environment. Normally, when compiling files, if the compiler encounters an object (such as an array), it can treat it as a constant and proceed. But in this case, not only is the environment object not constant, it has to be the same (i.e., EQ) object that was used in the interpreted function, because other functions may be closed over it too. It may be unreasonable to make the compiler deal with this special case. Also, compiled and interpreted code may use different representations of closed-over environments. In this case it would be impossible to compile an interpreted closure because other interpreted functions might be sharing the same environment. Tim -- Tim Moore moore@cs.utah.edu {bellcore,hplabs}!utah-cs!moore "Ah, youth. Ah, statute of limitations." -John Waters
yeh@cs.purdue.EDU (Wei Jen Yeh) (04/03/91)
Hello again,
Regarding the wrapper function that I asked, this is the final version that
I used. (Thanks again to Barry and the person for pointing out using
multiple-value-prog1.)
(defun monitor_1fun (fname)
(if (fboundp fname)
(if (get fname 'monitored)
(format T "Function ~A is already being monitored." fname)
(let ((old_fun (symbol-function fname)))
(setf (get fname 'monitored) old_fun)
(setf (symbol-function fname)
#'(lambda (&rest args)
(start_bench fname)
(multiple-value-prog1 (apply old_fun args)
(end_bench fname))))
; (compile fname)
(setq *Monitor_List* (cons fname *Monitor_List*))
*Monitor_List*))
(format T "function ~A is not defined." fname)))
However, the commented compile call failed. Is there any reason (both
in theory/implementation) why a lambda-closure cannot be compiled? Why
doesn't the standard (ep, ip) trick work?
I'm looking forward to the wisdom from you guys...
Wei Jen Yeh yeh@cs.purdue.edu
Department of Computer Science
Purdue University
West Lafayette, Indiana
--
Wei Jen Yeh yeh@cs.purdue.edu
Department of Computer Science
Purdue University
West Lafayette, Indianayeh@cs.purdue.EDU (Wei Jen Yeh) (04/04/91)
Thanks for the people replying my question.
Most of them mentioned that lambda-closures are not required to be compilable
under the (not yet official) 2nd language edition. Tim gave some concrete
reasoning why it is so. And contrary to what I thought, the #'lambda
definition WAS compiled when monitor_1fun got compiled. My original
thought is that it's not compiled because it's like an constant expression
and the behavior of the outer function does not include that of the inner
lambda expression. But I'm certainly glad I was wrong. BTW, where did you
guys get that info? is it covered in Steele's book?
Thanks again.
Wei Jen Yeh yeh@cs.purdue.edu
Department of Computer Science
Purdue University
West Lafayette, Indiana
--
Wei Jen Yeh yeh@cs.purdue.edu
Department of Computer Science
Purdue University
West Lafayette, Indianajinx@zurich.ai.mit.edu (Guillermo J. Rozas) (04/04/91)
It should be obvious why compiling a closed "lambda expression" (as if
there was any such thing) doesn't work; same reason that they can't be
EVAL'ed.
There are several problems that come up when compiling interpreted
closures. The biggest one I can think of is dealing with the object
that represents the closed-over environment. Normally, when compiling
files, if the compiler encounters an object (such as an array), it can
treat it as a constant and proceed. But in this case, not only is the
environment object not constant, it has to be the same (i.e., EQ)
object that was used in the interpreted function, because other
functions may be closed over it too. It may be unreasonable to make
the compiler deal with this special case.
Also, compiled and interpreted code may use different representations
of closed-over environments. In this case it would be impossible to
compile an interpreted closure because other interpreted functions
might be sharing the same environment.
Although I understand why CL does not require this to work, the issues
are not that hard to solve.
The MIT Scheme compiler provides an entry point, COMPILE-PROCEDURE
that given an interpreted closure will return a compiled closure that
behaves the same way.
Since MIT Scheme has first-class environments, we had to deal with the
situation of mutable environments being shared between interpreted and
compiled code, and directly manipulated by programs, and it is not
hard to solve, but requires some care.barmar@think.com (Barry Margolin) (04/05/91)
In article <JINX.91Apr3230349@chamarti.ai.mit.edu> jinx@zurich.ai.mit.edu writes: >Since MIT Scheme has first-class environments, we had to deal with the >situation of mutable environments being shared between interpreted and >compiled code, and directly manipulated by programs, and it is not >hard to solve, but requires some care. No, it's not hard to do, but few CL implementations actually do it, so we decided not to require it. Basically, all it requires is that the compiler generate code that emulates the way the interpreter searches lexical environments, rather than turning lexical references into constant offsets into the lexical environment. Also, the compilation of GO and RETURN-FROM must call into the interpreter if they are transfering out of the lambda expression. To answer Yeh's question about how Tim knew this: he is on the X3J13 committee, which is currently drafting the ANSI Common Lisp standard. The decision of the committee is mentioned in CLtL 2nd edition, on p.677, paragraph 3. -- Barry Margolin, Thinking Machines Corp. barmar@think.com {uunet,harvard}!think!barmar
jinx@zurich.ai.mit.edu (Guillermo J. Rozas) (04/05/91)
In article <1991Apr5.001438.26057@Think.COM> barmar@think.com (Barry Margolin) writes: Path: ai-lab!mintaka!think.com!barmar From: barmar@think.com (Barry Margolin) Newsgroups: comp.lang.lisp Date: 5 Apr 91 00:14:38 GMT References: <14172@medusa.cs.purdue.edu> <1991Apr3.095336.29725@hellgate.utah.edu> <JINX.91Apr3230349@chamarti.ai.mit.edu> Sender: news@Think.COM Organization: Thinking Machines Corporation, Cambridge MA, USA Lines: 23 In article <JINX.91Apr3230349@chamarti.ai.mit.edu> jinx@zurich.ai.mit.edu writes: >Since MIT Scheme has first-class environments, we had to deal with the >situation of mutable environments being shared between interpreted and >compiled code, and directly manipulated by programs, and it is not >hard to solve, but requires some care. No, it's not hard to do, but few CL implementations actually do it, so we decided not to require it. Basically, all it requires is that the compiler generate code that emulates the way the interpreter searches lexical environments, rather than turning lexical references into constant offsets into the lexical environment. Also, the compilation of GO and RETURN-FROM must call into the interpreter if they are transfering out of the lambda expression. As I said in my previous message, I understand why CL did not do it. I was just trying to say that it is not hard. I'm not sure that I disagree with the decision for CL. BTW, as an aside, your solution works but is not very good, since it makes compiled code access to these variables slow. There are alternatives which make compiled code run faster. What MIT Scheme does by default is as follows (in rough lines): The runtime library provides a way for compiled code to request a cell for a <symbol,environment> pair. It can then cache the cell, and reference (and assign) the variable by indirecting through the cell. The runtime library maintains a table of where such cells have been provided, and replaces them when incremental definition shadows previous definitions. The interpreter has been taught (trivially) to handle such cells. Thus when compiling any code, the way that top-level free variables are handled is by requesting cells for all of them at link-time, caching them in the compiled code object, and forgetting where they came from.
jeff@aiai.ed.ac.uk (Jeff Dalton) (04/10/91)
In article <1991Apr3.095336.29725@hellgate.utah.edu> moore%defmacro.utah.edu@cs.utah.edu (Tim Moore) writes: > >"It is an error if the function to be compiled was defined >interpretively in a non-null lexical environment. (An implementation >is free to extend the behavior of COMPILE to compile such functions >properly, but portable programs may not depend on this capability.)" > >It should be obvious why compiling a closed "lambda expression" (as if >there was any such thing) doesn't work; same reason that they can't be >EVAL'ed. I don't quite understand the point about EVAL. The original question talked about a "lambda-closure", which is presumably what you get as the value of (FUNCTION (LAMBDA ...)) in some Common Lisp. Indeed, in KCL you will get a list (LAMBDA-CLOSURE ...). [That sort of result isn't allowed in CLtL II, where functions are supposed to be distinct from lists, but it's allowed in CLtL I.] That is, it's a function object. It may make sense to say it can't be evaluated, just as it may make sense to say hash tables (for example) can't be evaluated. It isn't what we normally think of as an expression. But it is a function, and functions are just the sort of thing one can give to COMPILE. So I don't think it's _obvious_ why compiling them doesn't work, though it's certainly understandable -- for more or less the reasons you give below. >There are several problems that come up when compiling interpreted >closures. The biggest one I can think of is dealing with the object >that represents the closed-over environment. Normally, when compiling >files, if the compiler encounters an object (such as an array), it can >treat it as a constant and proceed. But in this case, not only is the >environment object not constant, it has to be the same (i.e., EQ) >object that was used in the interpreted function, because other >functions may be closed over it too. It may be unreasonable to make >the compiler deal with this special case. I think this is right except for a few details. When compiling a file, the compiler or loader can cause constants to be "coalesced" (see section 25.1.4 Similarity of Constants, and especially page 694, in CLtL II). However, COMPILE is not allowed to do this. An object referred to as a constant by the compiled code has to be the same (in this case EQL) object referred to by the code before it was compiled (see CLtL II, page 115). Consequently, if an environment were an object, it's arguable that it would be the usual case for COMPILE to make the compiled function use the same one as the interpreted function, not a special case. However, these environments are not 1st-class objects in Common Lisp, and there is no defined way for them to appear as constants in code, so the rule does not apply. >Also, compiled and interpreted code may use different representations >of closed-over environments. In this case it would be impossible to >compile an interpreted closure because other interpreted functions >might be sharing the same environment. It depends on how different the representations are. For example, in KCL two different representations are used in compiled closures. One of them is more or less the same as that used in interpreted closures (an alist), the other is not (it's a vector). Both representations might be used for the same environment, and it works because both will contain the same bindings. (If we say a variable names a location in which a value is stored, two functions closed over the same variables have to acess the same locations so that an assinment in one will be seen in the other. In KCL, the two representations contain the same locations.) So it doesn't matter if the environment representations are different so long as they contain the same locations for the variables. On the other hand, this is still a strong limitation on what representations could be used, and it would probably be unreasonable to impose it on implementations. -- jeff
moore%defmacro.utah.edu@cs.utah.edu (Tim Moore) (04/10/91)
In article <4448@skye.ed.ac.uk> jeff@aiai.UUCP (Jeff Dalton) writes: >In article <1991Apr3.095336.29725@hellgate.utah.edu> moore%defmacro.utah.edu@cs.utah.edu (Tim Moore) writes: >> >>"It is an error if the function to be compiled was defined >>interpretively in a non-null lexical environment. (An implementation >>is free to extend the behavior of COMPILE to compile such functions >>properly, but portable programs may not depend on this capability.)" >> >>It should be obvious why compiling a closed "lambda expression" (as if >>there was any such thing) doesn't work; same reason that they can't be >>EVAL'ed. > >I don't quite understand the point about EVAL. The original question >talked about a "lambda-closure", which is presumably what you get as >the value of (FUNCTION (LAMBDA ...)) in some Common Lisp. Indeed, in >KCL you will get a list (LAMBDA-CLOSURE ...). [That sort of result >isn't allowed in CLtL II, where functions are supposed to be distinct >from lists, but it's allowed in CLtL I.] I brought up "closed lambda expression", i.e. a lambda expression with free variables (not (FUNCTION (LAMBDA ...)), just (LAMBDA ...)), to cover all bases. There seemed to be some confusion about terminology. > >>[Tim said the compiler may have trouble dealing with shared environments] > >I think this is right except for a few details. > >When compiling a file, the compiler or loader can cause constants to >be "coalesced" (see section 25.1.4 Similarity of Constants, and >especially page 694, in CLtL II). However, COMPILE is not allowed to >do this. An object referred to as a constant by the compiled code has >to be the same (in this case EQL) object referred to by the code >before it was compiled (see CLtL II, page 115). > >Consequently, if an environment were an object, it's arguable that it >would be the usual case for COMPILE to make the compiled function use >the same one as the interpreted function, not a special case. > You are correct. Just because our compiler is broken in this case doesn't mean everyone's is :-) -- Tim Moore moore@cs.utah.edu {bellcore,hplabs}!utah-cs!moore "Ah, youth. Ah, statute of limitations." -John Waters